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ABSTRACT 


There are millions of police, fire, public health, emergency medical, emergency 
management, and public sector homeland security professionals ready and willing to 
assist in the global war on terror (GWOT) and current strategies to strengthen homeland 
security include the provision of unclassified intelligence products to these non- 
traditional recipients (NTR). Simply pushing intelligence products to NTR is not enough, 
NTR must possess adequate contextual background in order to effectively utilize 
intelligence provided by the Intelligence Community (IC) in implementing strategies in 
information driven and risk based prevention and response. Given the diversity of NTR, 
distribution of “one size fits all” products ensures that the intelligence will fit no one’s 
needs. 

This thesis researches the impact of intelligence contextual background fusion 
(CBF) through the use of hyperlink technology and evaluates the likelihood of hyperlink 
acceptance by NTR. By utilizing DHS and FBI customer satisfaction survey questions in 
“quality” and “value” factors along with previously validated Technology Acceptance 
Model (TAM) questions in “ease of use” and “usefulness” factors, this research finds that 
CBF significantly improves both perceived value and quality, and finds that NTR 
overwhelmingly prefer a CBF product. NTR broadly accepted hyperlink technology in 
this application. 
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I. INTRODUCTION 


A. PROBLEM STATEMENT 

Multiple initiatives, including the Homeland Security Information Network 
(HSIN) and the Joint Regional Information Exchange System (JRIES), as well as 
supporting directives such as National Capability Specific Priority 3.2.1: Strengthening 
Infonnation Sharing and Collaboration Capabilities, have ensured that non-traditional 
recipients (NTR) in law enforcement, fire, emergency medical services (EMS), 
emergency management, public health and the private sector are receiving unclassified 
intelligence products from multiple sources including the Department of Homeland 
Security (DHS), the Federal Bureau of Investigation (FBI), Fusion Centers, Joint 
Terrorism Task Forces (JTTF), and Terrorism Early Warning Groups (TEWG). 
However, simply pushing intelligence products to non traditional recipients (NTR) is not 
enough. Recipients must possess enough contextual background to effectively utilize the 
intelligence products in order for this strategy to be successful. Unclassified intelligence 
distribution must be useful, easy to use, deliberate and coordinated, maximizing 
recipients’ ability to effectively utilize the intelligence, regardless of the time of day, day 
of week, or location of the recipient. We can do better in this regard. As discussed in a 
Markle Foundation Working Group Report, Networking of Federal Government 
Agencies with State and Local Government and Private Sector Entities, “...adequate 
context for homeland security providers to effectively utilize infonnation is specific, 
tailored for each local entity, rapidly disseminated, and does not overburden recipients 
with vague or irrelevant information.” 1 The Final 9/11 Commission Report noted the 
importance of context in decision making, reporting that the President was provided 
intelligence “news without... much context” prior to September 11, 2001, contributing to 
a failure of decision makers to recognize that Bin Laden posed a “novel danger.” 2 A 

1 Zoe Baird, Creating a Trusted Network for Homeland Security (New York, New York: The Markle 
Foundation, 2003), 56, 

http ://www.markle.org/downloadable_assets/nstf_report2_part_two.pdf#search=%22adequate%20contextu 
al%20background%20homeland%20security%22 (accessed September 24, 2006). 

2 Thomas Kean et al. The 9/11 Commission Report. (Washington, D.C.: United States General 
Accounting Office, 2004), 119, 

http://files.fmdlaw.com/news.fmdlaw.com/hdocs/docs/91 lfinalrpt/911 report.pdf. (accessed January 27, 
2006). 
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similar lack of context in the evaluation of Iraq intelligence assessments contributed to a 
failure to understand Saddam Hussein’s perception of the outside world. 3 At the local 
level, in 2005, Police Chief Patrick Miller found that “There is profound belief amongst 
law enforcement leaders in California that the essential mechanisms for information 
sharing are poor at best and arriving at some consensus on how to fix that problem was 
far more important than examining leadership models and future impacts.” 4 The 2007 
Department of Homeland Security Appropriations Act reinforces this charge for the 
Department of Homeland Security (DHS), documenting their responsibility to 
“...develop better methods for the sharing of intelligence with State and local law 
enforcement agencies...” with similar concerns, the General Accounting Office has 
identified at least 11 separate information sharing authorities and initiatives since 9/11. 5 > 6 

These findings and directives make it clear that one mission of the federal 
government is to effectively involve all public safety providers in homeland security 
through improved information sharing. Effective involvement of NTR would in part 
result in an improved disaster response system, reduction of man made disasters, and 
“fusion” of emergency response, preparedness, recovery, mitigation, and critical 
infrastructure protection. 7 Critical decision making is not limited to Chiefs of Police, 
Fire or Public Health, but includes professionals at every level of organizations. In fact, 
those “on the street,” maintain the most public contact, and as such, make some of the 
most critical homeland security decisions. While the media and public may focus on 
police chiefs, command officers, detectives, or other high visibility law enforcement 

3 Christopher Andrew, “Intelligence Analysis Needs to Look Backwards before Looking Forward,” 
History’ and Policy. org, 1, http://www.historyandpolicy.org/archive/pol-paper-print-23.html (accessed May 
19,2006). 

4 Patrick Miller, How Can We Improve Information Sharing Among Local Law Enforcement 
Agencies, Master’s Thesis, Naval Postgraduate School, Monterey, California, March 2005. 

5 Department of Homeland Security’ Appropirations Act, 2007, H.R. 5441, 109, 2, July 17, 2006, 
S7632, https://www.chds.us/courses/file.php/112/moddata/fomm/742/13414/DHS_Approp_Bill.txt 
(accessed August 22, 2006). 

6 Government Accountability Office, “Information Sharing the Federal Government Needs to 
Establish Policies and Processes for Sharing Terrorism-Related and Sensitive but Unclassified 
Information,” United States Government Accountability Office Report to Congressional Requestors 6, no. 
385 (March 17, 2006): 9, http://www.gao.gov/new.items/d06385.pdf (accessed May 25, 2006). 

7 DEPARTMENT OF HOMELAND SECURITY APPROPRIATIONS ACT, 2007, H.R. 5441, Senate 
Debate on the Department of Homeland Security Appropriations Act 2004, 1. 
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positions in homeland security, patrol officers have consistently been shown to have the 
greatest impact on crime. As described by Captain Dennis Potter, Operations 
Commander for the Columbine school shootings, “When something really bad happens, 
the initial response strategies are developed from the front seat of a patrol car, not from a 
command officers desk...” 8 Captain Potter’s observation reflects an understanding that 
intelligence is critical in day to day tactical decision making, not just in long term 
strategic planning. Research documents what police executives kn ow about the critical 
nature of street level operations, finding that 80% of crimes are solved by patrol 
operations, not detective work. 9 Whether planning long term strategic or short term 
tactical operations, it is critical to ensure that street level decision makers have the 
intelligence they need. This is the audience of unclassified intelligence operations, and 
while it may not be practical to provide classified intelligence to these millions of 
professionals, it is imperative that they are provided the very best unclassified 
intelligence possible. 

This research does not attempt to detennine whether providing classified or 
unclassified intelligence to NTR is an effective strategy. The choice to share unclassified 
intelligence products has already been made; this research seeks to identify the impact of 
implementing contextual background fusion (CBF) with intelligence already provided to 
NTR on the perceived value and quality of that intelligence. Bill Nolte, Director of 
Education and Training, Office of the Director of National Intelligence, reflected on the 
need to ensure intelligence usefulness, “We are not in the secrets business; we are in the 
information business... The buggy whip manufacturer who survived realized he was in 
the transportation business.” 10 This research goes further, contending that we are in 
neither the secrets, nor information business, but the decision-making business. If 
intelligence products do not improve decision making, the IC might as well produce 

8 Dennis Potter, The First Three Minutes: Law Enforcement’s Role in Crisis Management (Fairfax, 
Virginia: Commission on Accreditation for Law Enforcement Agencies, 2006), 1, 

http://64.233.187.104/search?q=cache:0Y6GlXiUjgkJ: www.calea.org/newweb/newsletter/No82/first3minu 
tes.htm+dennis+potter+the+first+three+minutes&hl=en&gl=us&ct=clnk&cd=l. (accessed October 7, 

2006). 

9 Tom O’Conner and Paula Baker, Police in Society Syllabus. (Raleigh, North Carolina: Wesleyan 
College, 2005), 1, http://faculty.ncwc.edu/toconnor/205/2051ect08.htm (accessed October 18, 2006). 

10 Robert Nolte, “Remarks to Center for Flomeland Defense and Security Cohorts 0503, 0504“ 
(lecture. Naval Postgraduate School, Monterey, February 27, 2006). 
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buggy whips. This research addresses the need for contextual background by decision 
makers, as well as the needs of the IC, who depend upon continued funding by decision 
makers. 

Policy demands that more information be shared; for that information to be 
effectively utilized, decision makers must perceive both value and quality in that 
information. 11 This research does not fully address defining the needs of non-traditional 
intelligence recipients, another critical opportunity for research, but narrowly focuses on 
the impact of CBF of open source information sources onto unclassified intelligence 
through hyperlink technology. The need for contextual background is evident at every 
level of decision making; in confirmation hearings, National Director of Intelligence 
nominee General Hayden reflected on lack of context in decision making in response to 
questioning by Senator Feinstein: “One key one that I wanted to mention when the 
chairman was talking about it, the Iraq WMD estimate was essentially worked in a WMD 
channel. It was absent a regional or cultural context. We are not doing that 
now.. .We’re not doing that on Iran.” 12 (Emphasis added) 

Understandably, in the midst of terrorist attack, when contextual background is 
most needed by decision makers, those with personal knowledge of relevant context and 
background are least available to the homeland security professionals in the 
approximately 18,000 law enforcement jurisdictions who may seek the information, let 
alone other homeland security disciplines; professionals who will use whatever context 
and background is available to them for their decision making. As it was on 9/11, police 
chiefs, fire chiefs, private security, emergency medical services and public health 
leadership should not expect return phone calls from any federal or state agency that 

11 Maryam Alavi, “An Assessment of the Concept of Decision Support Systems as Viewed by Senior- 
Level Executives,” MIS Quarterly 6, no. 4 (December 1982): 1-9 

http://www.jstor.org.libproxy.nps.navy.mi1/view/02767783/ap010024/01a00040/77searchUrHhttp%3a//ww 
wjstor.org/search/BasicResults%3fhp%3d25%26si%3dl%26Query%3dperceived%2bvalue%2bin%2bdeci 
sion%2bmaking&frame=noframe&currentResult=02767783%2bap010024%2b01a00040%2b0%2cFF03& 
userID=837833bl@nps.navy.mil/01cc99333cb9210c8aec98ea&dpU3&config=jstor. (accessed July 21, 
2006). 

12 Senate Select Committee on Intelligence, Hearing of the Seriate Select Committee on Intelligence 
on the Nomination of General Michael Hayden to be the Director of Central Intelligence, 109-112, May 
16, 2006, 75-76, http://www.fas.org/irp/congress/2006_hr/051806transcript.pdf (accessed September 5, 
2006). 
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might be able to provide contextual background, as these agencies will be inundated by 
requests and operational support requirements. When information is most needed, 
individuals with the contextual background will be least available, busy instead 
supporting their home agencies in crisis management. Homeland Security professionals 
looking for contextual background are left to their own resources to obtain the 
information they need, potentially from questionable Internet sources. 

In one effort to emphasize the importance of context, the U.S. Government 
Printing Office/CQ Congressional Reporting Service citations include the phrase 
“Providing government documents on demand, in context” explicitly acknowledging the 
critical relationship between infonnation sharing and context. Intelligence producers 
must also provide intelligence “on demand, in context,” unequivocally affirming the 
relationship between effective information sharing and context. 

Providing unclassified intelligence “on demand, in context” is critical for millions 
of employees in NTR disciplines who incorporate that context into day to day decision 
making in public contacts, policy development, strategy and tactics. In the absence of 
CBF by intelligence producers, open Internet searches of unvetted sites by intelligence 
recipients can prove not only unreliable, but entirely inaccurate. For instance, a Google 
search that might be completed by a homeland security professional searching for 
contextual background on aircraft use in Islamic terrorist attacks, “Islamic terrorists 
kamikazes weapon aircraft” leads to a “Non Aligned Press Network” story where it is 
reported that the planes on 9/11 were flown utilizing remote controls by individuals in 
American government. False Internet postings such as this are common enough that the 
U.S. State Department attempts to identify such misinformation on 
USINFO.STATE.GOV. 13 This webpage highlights the danger of utilizing unvetted, open 
searches for contextual background; If the intelligence producer fails to provide CBF 
with their product, NTR may find very authentic looking information on the Internet that, 
when combined with timely, accurate and actionable intelligence, produces poor 
decisions. 

13 United States State Department, Did a Plane Hit the Pentagon? (Washington, D.C.: Department of 
State, 2005), 1, http://usinfo.state.gov/media/Archive/2005/Jun/28-581634.html (accessed September 15, 
2006). 
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The context and background utilized by decision makers must be reliable, vetted 
information that is perceived as high quality and valuable, and it must be consistent with 
information being utilized by the Department of Homeland Security (DHS), the Federal 
Bureau of Investigation (FBI), the military, and other state, local and federal homeland 
security professionals. Through CBF, the IC can ensure that contextual background used 
NTR is reliable, vetted, and consistent with the henneneutic of the producing agency. 
The IC must know if incorporating a CBF system with intelligence products provided to 
NTR will also improve the perceived value and quality of that intelligence. 


B. SPECIFIC RESEARCH OBJECTIVE 

The overall objective of this research is to improve the United State’s ability to 
identify and mitigate terrorist threats by examining the impact of CBF on the perceived 
value and quality of intelligence products. DHS and FBI provide unclassified 
intelligence to NTR and routinely request feedback on perceived value and quality as part 
of the intelligence cycle focused on continual counter terrorism improvement. If CBF is 
shown to improve DHS/FBI defined value and quality factors, the technology utilized 
must be evaluated for technology acceptance by NTR. Technology Acceptance Model 
research has shown that technology must be perceived as both easy to use and useful if it 
is to be accepted for widespread use. Tested, reliable, and generally accepted technology 
such as hyperlinks allow the fusion of related documents in a manner that supports a user 
defined experience within linking constraints established by the author. Context and 
background to intelligence products must meet recipient defined needs. As noted by the 
Major Cities Police Chiefs Association in a 2002 report, 

...our first line of defense against terrorism—the seven hundred thousand 
officers on the street— (must) be given adequate training and background 
information on terrorism...our local and state officers should have 
background knowledge...officers should have the ability to access 
national data banks... 14 


14 Edward Tully, Terrorism the Impact on State and Local Law Enforcement (Alexandria, VA: 
International Association of Chiefs of Police, 2002), 5, 

http://www.neiassociates.org/mccintelligencereport.pdf (accessed September 2, 2006). 
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This research is designed to detennine the impact of fusing the background 
information demanded by the Major City Police Chiefs on the perceived value and 
quality of intelligence products for not just law enforcement, but for the millions of 
homeland security professionals in fire, EMS, public health, emergency management, and 
the private sector. If CBF results in improved value and quality, technology acceptance 
model (TAM) research provides a framework for the evaluation of the potential usage of 
such a system; likelihood of use in this model is influenced by “perceived ease of use,” 
and “perceived usefulness.” 15 This research applies previously validated TAM processes 
to evaluate the likelihood that NTR will utilize hyperlink technology in CBF applications. 


C. SIGNIFICANCE OF RESEARCH 

Allied non-state actors in the private sector control approximately 85% of the 
nation’s critical infrastructure in the over 87,000 different U.S. jurisdictions. 16 For every 
federal law enforcement officer, there are approximately five public sector health 
professionals, seven state/local law enforcement officers, ten firefighters and twenty-one 
private security professionals, along with countless other public works, emergency 

17 18 19 

management and emergency medical professionals. ’ ’ As of June 2000, there were 
approximately 708,022 state and local law enforcement officers in the United States. 20 
As of June 2002, there were only 93,000 Federal law enforcement officers, or less than 
12% of sworn law enforcement. 21 The FBI, lead investigative agency for domestic 
terrorism, had only 12,416 agents as of October 2005, or approximately 1.5% of total law 

15 F. D. Davis, R. Bagozzi and P. R. Warshaw, “User Acceptance of Computer Technology,” 
Management Science 35, no. 8 (1989), 985, http://links.jstor.org/sici7sicU0025- 
1909(198908)35%3A8%3C982%3AUAOCTA%3E2.0.CO%3B2-1 (accessed May 26, 2006). 

16 George Bush, The National Security Strategy of the United States of America (Washington, D.C.: 
Government Printing Office, 2002), vii. 

17 State and Local Law Enforcement Statistics, (Washington, D.C.: Bureau of Justice Statistics, 2006). 
1, http://www.ojp.usdoj.gov/bjs/sandlle.htm (accessed December 1, 2005). 

18 Federal Law Enforcement Statistics , (Washington, D.C.: Bureau of Justice Statistics, 2006). 1, 
http://www.ojp.usdoj.gov/bjs/fedle.htm (accessed December 2, 2005). 

111 Association of State and Territorial Health Professionals, State Public Health Worker Employee 
Shortage Report, A Civil Service Recruitment and Retention Crisis (Washington, D.C.: ASTHO.org), 3. 

20 Federal Law Enforcement Statistics, Bureau of Justice Statistics, 1. 

21 Ibid. 
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enforcement. In order to maximize our defense against asymmetric threats, we must 
effectively utilize unclassified intelligence to educate these diverse professionals on 
threats, engage them in the intelligence process, and enlist them to provide information to 
federal partners so that appropriate preventative measures can be considered. We must 
improve our performance to meet this need. Attorney General Alberto Gonzalez, 
speaking before the Council on Foreign Relations in New York, remarked on December 
1, 2005, “we must fight an intelligent war against terrorism using every tool 
available...This is the government’s obligation, and it is the American people’s 
expectation... it is your expectation.” 22 If we are to use “every tool available,” and “fight 
an intelligent war...” then we must educate and utilize the 98.5% of law enforcement 
that are not FBI agents along with other NTR professionals in the asymmetric conflict 
threat posed by hostile non-state actors. If found to be effective, CBF for NTR would 
represent a concrete, visible step in transformation from a need-to-know to a need-to- 
share culture. 

Classified intelligence needs of local law enforcement are being addressed 
through Joint Terrorism Task Forces (JTTF) and the development of state and local 
fusion centers, some of which include fire service and private sector partners. Classified 
and unclassified information can be shared with participants who have received security 
clearances. One of the limitations of these centers is the need for those directly involved 
with the centers to live in the geographic region of the centers; in a state such as 
California that covers a vast geographic area, this can be a significant limitation, even 
with multiple fusion centers within the state. It is clear that technological assistance will 
be required to reach a critical mass of professionals over a wide area in a timely manner. 
As noted in the Markle report, “Protecting America’s Freedom in the Information Age,” 

The DHS should become the base for building up a national community of 
intelligence contributors and analysts. To create a national infrastructure 
that is aware, robust, and resilient to the many challenges we face in the 
21st century, we have to harness the power and dynamism of information 


22 Alberto R. Gonzales, Prepared Remarks for Attorney General Alberto R. Gonzales 
(lecture, Council on Foreign Relations, New York, December 1, 2005), 1, 
http://www.usdoj.gov/ag/speeches/2005/ag_speech_051201.html (accessed December 4, 2005). 
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technology by utilizing the strengths and mitigating the weaknesses of our 
networked society. 23 (Emphasis Added) 

CBF intelligence must utilize infonnation technology with an ability to reach over 
87,000 jurisdictions, while also maintaining integration with other Federal, State and 
Regional efforts. The implications to homeland security are significant. The adage, 
“Quantity has a quality all its own” reflects the potential impact of the improved use and 
involvement of NTR in the intelligence process. Small expenditures in improving 
unclassified intelligence are leveraged by the millions of diverse NTR along with the 
billions of dollars expended to date in community outreach programs such as community 
oriented policing (COP). An estimated 7.5 billion dollars was expended for COP alone 
from 1994-2000. 24 

Significant efforts have been focused on intelligence analysis and upward 
intelligence flow; stimulating NTR involvement in intelligence operations through 
improved dissemination can exponentially increase the quantity and quality of that 
upward intelligence flow. Focusing readily available technology, existing infrastructure, 
and intelligence smart practices on NTR will result in improved information flow, 
supporting analytical success and initiating an upward spiral of intelligence quality and 
effective counter-terrorism operations. As noted in the Department of Army 
Counterinsurgency Manual, Final Draft, 

Intelligence and operations feed back on one another. Effective 
intelligence drives effective operations, which produce more intelligence. 
Similarly, ineffective or inaccurate intelligence produces ineffective 
operations, which reduce the availability of intelligence. 25 


23 Zoe Baird, Protecting America's Freedom in the Information Age (New York, New York: The 
Markle Foundation, 2002), 15, http://knxup2.ad.nps.navy.mil/homesec/docs/nonprof/nps05-070903-01.pdf 
(accessed February 2, 2006). 

24 G. G. Davis and others, The Facts about COPS: A Performance Overview of the Community’ 
Oriented Policing Services Program (Washington, D.C.: Fleritage Foundation, 2000), 1, 
http://www.heritage.org/Research/Crime/CDA00-10.cfm (accessed January 12, 2007). 

25 Headquarters, Department of the Army, COUNTERINSURGENCY Draft Not for Implementation. 
(Washington, D.C.: Headquarters, Department of the Army, 2006), 3-2, 
http://www.fas.org/irp/doddir/army/fm3-24fd.pdf (accessed August 12, 2006). 
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D. REVIEW OF RELEVANT LITERATURE 


1. What is Contextual Background? 

In the study of philosophy, some believe the understanding of text is dependent on 
the context in which the reader exists. As the text is read, it is interpreted based on the 
social context and bias of the interpreter. Additionally, text has its own “horizon of 
meaning” which is influenced by the contextual background of the writer, the time of 
writing, and the originating context. Philosophical henneneutics examines the 
relationship between a reader and text, both of which must be understood within the 
context of their experience and creation respectively. 26 

Hermeneutics as a theory involves both the understanding and interpretation of 
text that has continued from the time of Plato. It has been commonly applied to both 
Biblical and cultural research. In “Teoria Della Interpretatione,” Emilio Betti argues that 
text is “objectified representations of human intentions,” and to grasp the true and only 
meaning of the text, a reader must understand the original process of creation. 27 In the 
creation and distribution of unclassified intelligence, hermeneutic fusion is problematic; 
the original process of creation is often based on classified intelligence, which may not be 
fused to an unclassified document. As the author/producer of unclassified intelligence is 
prohibited from the fusion of classified henneneutic material, open source contextual 
background chosen by authors/producers must be used to replace the original, classified 
material that influenced the hermeneutic of the writer at the time of creation. In this 
research, contextual background is defined as unclassified material that best represents 
the hermeneutic of the producer at the time of creation. 

2. Contextual Background and Intelligence Usefulness 

Discussions about intelligence usefulness have been focused primarily on 
traditional, classified, intelligence recipients in the government. As intelligence agencies 
move from a “right to know” to a “right to share” culture and more NTR begin to receive 
unclassified intelligence products, intelligence producers must focus on intelligence 

26 Frank Ravitch, “Struggling with Text and Context: A Hermeneutic Approach to Interpreting and 
Realizing Law School Missions,” St. Johns Law Review (2000), 5, 

http://fmdarticles.eom/p/artieles/mi_qa3735/is_200007/ai_n8886993/pg_5 (accessed 10, 2006). 

27 Stanford Encyclopedia of Philosophy, Henneneutics (Palo Alto, CA: Stanford Encyclopedia of 
Philosophy, 2005), 1, http://plato.stanford.edu/entries/hermeneutics/ (accessed January 12, 2007). 
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usefulness for NTR. As expressed by John Hillen in the National Review, “U.S. 
Intelligence Failures stem from too much infonnation, not enough understanding.” 28 
This is a subtle, but critical point; even a high quality intelligence product that is not fully 
understood by NTR may lack usefulness, failing to provide the knowledge required to 
improve decision making. There is agreement that ultimately, it is the usefulness of the 
intelligence product in achieving homeland security that matters. 

The sentiment of Hillen is echoed in CIA reports, outlining that the provision of 
contextual background is a critical component in the usefulness of intelligence products. 

In periods of crisis, when demands are high and response time is short, 
most written intelligence production is in the form of policy-driven memos 
and briefs and pieces written for daily publications. The result of this 
narrowly focused and piecemeal intelligence flow is that it does not foster 
continuity of analysis nor does it provide a context within which to place 
seemingly unrelated information. In the case of Iraq, national intelligence 
did not provide a comprehensive picture of how the country functioned as 
a whole. The Intelligence Community has made substantial, although 
sporadic, efforts over the past decade and a half to explore better and 
more technologically advanced methods of communicating with 
consumers. The results, however, have been modest at best. The 
requirement to have background and contextual information available 
at the policymaker's fingertips in a timely fashion remains unfulfilled. 29 
(Emphasis added) 


Decision makers often demand “tailored” intelligence briefs beyond what is 
prepared for general distribution in order to meet their decision making needs. 30 This 
issue is exacerbated by the recent addition of NTR. 31 These recipients do not have 
personal intelligence analysts to produce “tailored” briefs that contain necessary 
contextual background. Additionally, NTR are often unaccustomed to the intelligence 

28 John Hillen, “Know Nothings: U.S. Intelligence Failures Stem from Too Much Information, Not 
enough Understanding,” National Review, 8/ 3. (1998), 1, 

http://www.fmdarticles.eom/p/articles/mi_ml282/is_nl4_v50/ai_21102283 (accessed January 12, 2007). 

29 Kerr et ah. Intelligence and Analysis on Iraq: Issues for the Intelligence Community, 1, 
http://www.gwu.edu/~nsarchiv/news/20051013/kerr_report.pdf. (accessed January 24, 2006). 

30 Peter Pirolli, Assisting People to Become Independent Learners in the Analysis of Intelligence (Palo 
Alto, California: Office of Naval Research, 2006), 38, http://www.fas.org/irp/eprint/pirolli.pdf . (accessed 
September 2, 2006). 

31 Anonymous, Northern California Regional Terrorism Threat Assessment Center, 
http://www.ncrttac.org/ (accessed January 24, 2006), 1. 
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cycle and use of intelligence products as their discipline related training and experience 
did not previously require the use of such intelligence. As described by Lisa Palmieri, 
President of the International Association of Law Enforcement Intelligence Analysts, 


I recall reading many articles in the media in which law enforcement 
executives demanded more “information sharing”, particularly focused at 
the FBI. I also recall thinking that if these executives got what they asked 
for, they would be buried in uncorroborated, unevaluated, “white 
noise”. At that time, very few state and local police departments had 
analytic capabilities, so being barraged with information would defeat the 
purpose of guiding law enforcement decisions. This has, unfortunately, 
come to pass, with law enforcement agencies erring on the side of caution; 
they are sharing more piece-meal information than could ever be made 
useful in case any small detail might possibly be deemed important in 
retrospect. (Emphasis added) 32 

Volumes of information without contextual background can overwhelm NTR and 
is not useful. 


3. Contextual Background and Decision Making 

There is agreement that contextual background is a critical component in 
intelligence decision making. 33 Extensive research exists on decision making theory, 
with similarly extensive discussion of the impact of intelligence products on decision 
making. Researchers in one camp of decision making theory define three different types 
of context that are relevant to decision making: 

• Proceduralized context which is shared by those involved in the problem and 
is directly but tacitly used for the problem solving. 

• Contextual knowledge that is not explicitly used but influences the problem 
solving. 

• External knowledge that has nothing to do with the current decision making 
but is kn own by many of those involved. 


32 Lisa M. Palmieri, Information vs. Intelligence: What Police Executives Need to Know 
(Massachusetts: 1ALEIA, 2005), 1, http://www.ialeia.org/pubs/InformationvsIntelligence.pdf (accessed 
January 28, 2006). 

33 H. Bradford Westerfield, “Inside Ivory Bunkers; CIA Analysts Resist Managers’ ‘Pandering’,” in 
Strategic Intelligence: Windows into a Secret World, ” ed. by Loch Johnson and James Wirtz (Los Angeles, 
California: Roxbury Publishing Company, 2004), 198, 

http://www.hsdl.org/homesec/docs/zroom/cm/ns4156/westerfield2004pl98.pdf (accessed February 2, 
2006). 
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Proeeduralized contexts 



Figure 1. Different types of context (From Brezillon and Pomerol, 1999) 

Context is defined as the “sum of all the knowledge possessed by the operators on 
the whole task.” 34 

In this Pomerol and Brezillon model, a decision maker would, when presented 
with an intelligence product, access his or her contextual knowledge, and then 
proceduralize that knowledge based on the intelligence product in hand; in this process, 
the decision maker may also access available external contextual information, 
incorporating that knowledge and proceduralizing it prior to decision making. As noted 
by Pomerol and Brezillon, “... it is clear that a Palestinian whose prior knowledge is 
reduced to his Imam’s preaches cannot have the same interpretation of Middle East 
events as a Harvard alumnus.” 35 This research addresses the impact of providing 
external contextual knowledge to decision makers. 


34 Patrick Brezillon and Jean-Charles Pomerol, Procedwalization of the Contextual Knowledge for 
Decision Making, Abstract (October 28, 2002), 3, http://www- 

poleia.lip6.fr/~brezil/Pages2/Publications/Luxembourg.pdf (accessed February 2, 2006). 

35 Patrick Brezillon and Jean-Charles Pomerol, Procedwalization of the Contextual Knowledge for 
Decision Making, Abstract (October 28, 2002), 3, http://www- 

poleia.lip6.fr/~brezil/Pages2/Publications/Luxembourg.pdf (accessed February 2, 2006). 
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Those involved in the practical application of decision making contend that 
“contextual analysis” of intelligence is vital. While many understand that intelligence 
intercepted “right from the enemy’s mouth” is necessary for accuracy, others argue that 
human intelligence sources add the required context to most fully understand and utilize 
information. 36 Both camps agree that context is important in the decision making process 
regardless of the source or nature of the infonnation; the practical, traditional model 
illustrated in the Figure below outlines the impact of the addition of context to both 
knowledge and understanding, impacting perceptions and decisions. 

A Traditional Model for Decision Making 



Figure 2. Context (From the North Atlantic Treaty Organization, 2002) 

Understandably, military applications of context in decision making center on 
pragmatic application. Training material utilized by the North Atlantic Treaty 
Organization (NATO) reflects that practical application of context in decision making: 
“...knowledge is produced when information is correlated with a model of the world and 

36 Protecting America’s Freedom in the Information Age , 48. 
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the current context.” 37 Regardless of whether context and decision making are evaluated 
through the lens of the academic or the war fighter, literature documents that context 
involved in the decision making process has a significant impact on the nature, quality 
and effectiveness of the decision. NTR of homeland security intelligence must have a 
mechanism to “...have background and contextual information available at the {their) 
fingerprints in a timely manner...” in order to most effectively utilize available 
intelligence. Contextual background is necessary so that NTR can make infonned 
decisions on protective actions and response plans. 

4. Non Traditional Recipients and Contextual Background 

The provision of contextual background for unclassified intelligence supplied to 
NTR is not well documented. While there is varied discussion on the need for contextual 
background regarding decision making, as well as the importance of context in military, 
intelligence and policy making decisions, the full integration of NTR in homeland 
security is still in the early stages of transfonnation - the provision of contextual 
background for NTR is no exception. The CEO of Cisco systems, John Chambers, 
highlighted this in a letter on behalf of the National Infrastructure Advisory Council to 
President G.W. Bush: “Full integration: Private industry must be fully integrated into the 
Federal Government’s Intelligence Cycle. The Federal Government must leam to include 
industry experts as domestic intelligence assets, integral to improving infrastructure 
protection, and not just occasional customers of government intelligence products.” 38 

Urban Area Security Initiatives (UASI) provide one solution in supplying 
homeland security contextual background to NTR. Some metropolitan areas may have 
the capacity in their limited geographic area to provide context and background for their 
intelligence through traditional workplace education; however, this capacity quickly 
diminishes the farther the recipient is from intelligence providers in both distance and 
discipline. Intelligence support of NTR outside UASI regions remains a gap in the 
current solutions set. Providing generalized intelligence education through the UASI 

37 North Atlantic Treaty Organization, Tactical Decision Aids and Situational Awareness 
(Amsterdam: North Atlantic Treaty Organization, 2002), 14, 

http://knxup2.ad.nps.navy.mil/homesec/docs/dtic/ADA400661.pdf (accessed January 27, 2006). 

38 Chambers and Nye, Letter to George Bush, Washington, D.C.; National Infrastructure Advisory 
Council, 1, http://www.dhs.gov/interweb/assetlibrary/NIAC_EEIS_Letter_0804.pdf (accessed January 24, 
2006). 
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network, including planning and direction, collection, processing, analysis, production 
and discrimination, begins the process of integrating NTR into the intelligence cycle. 39 
As NTR receive intelligence products relevant to their geographic area and discipline, the 
provision of relevant background information can provide context and a depth of 
understanding not otherwise readily available, providing decision makers with an 
increased capacity to effectively utilize the intelligence in protection of critical 
infrastructure and provision of leads back to the IC. 

In addition to the UASI resources available in some metropolitan areas, law 
enforcement NTR have technological options in obtaining contextual background for 
unclassified intelligence products not available to other NTR such as the Regional 
Information Sharing System (RISS) and Law Enforcement Online (LEO). Combined, 
firefighters and private sector security guards alone account for approximately three 
million NTR who currently lack access to these systems. 40 While law enforcement may 
employ these information systems, the contextual background contained in these data 
warehouses is not currently fused with intelligence products, requiring recipients to 
search for information. Although more effective than an open internet search (due to the 
validity an accuracy of the source information), accessing appropriate infonnation is still 
dependant on the varied skill, interest level and time available to each recipient - results 
may not reflect the hermeneutic of the producing agency. This research examines the 
impact of CBF to intelligence products for the millions of first responders who are 
currently without access to existing systems across America. Robert Steele highlights the 
value of context in the intelligence process for all recipients: 

New Value is in Content + Context + Speed The traditional craft of 
intelligence has tended to fragment content from its context, and be 
largely oblivious to timing. This is true both in the collection cycle and in 
the production cycle. The new craft of intelligence recognizes that the 
value of any given information, apart from its relevance to the decision at 
hand, stems from a combination of the content in context, and the 


39 Paul W. Parfomak, Guarding America: Security Guards and U.S. Critical Infrastructure 
Protection (Washington, D.C.: The Library of Congress: Congressional Research Service, 2004), 27, 
http://www.fas.org/sgp/crs/RL32670.pdf (accessed January 21, 2006). 

40 Ibid. 
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content in time. Both collectors and producers of intelligence must be 
acutely sensitive to the day-to-day needs of their consumers. 47 (Emphasis 
Added) 

Steele understands that the value of intelligence is not based solely on the sources 
utilized in the intelligence product, and that “historical knowledge,” (referenced by 
Pomerol and Brezillon as “contextual knowledge”) is a critical factor in the value of 
intelligence products. This research identifies the impact of avoiding “fracturing content 
from its context,” by fusing content (existing intelligence products) + context (vetted, 
valid and accurate) + speed (hyperlink fusion) for non traditional recipients. Speed 
through hyperlink fusion leverages the attention span of the reader, providing contextual 
background at the exact moment the issue is facing the reader. 

While contextual background is not specifically mentioned, the Office of the 
Director of National Intelligence is responsible for both traditional and non traditional 
recipient intelligence and “ensures that the Intelligence Community is integrated, 
focusing on the right questions, and maximizing the return on taxpayer dollars by 
identifying threats clearly so that policymakers, legislators, military commanders, and 
law enforcement officials can make well-informed decisions and take effective actions” 
(emphasis added). 42 The National Strategy for Homeland Security also provides 
evidence and summarizes the significance of providing high quality intelligence to NTR, 
citing a need to “Enable effective partnership with state and local governments and the 
private sector...which controls eighty-five percent of America’s infrastructure... 
Government at all levels must enable, not inhibit, the private sector’s ability to carry out 
its protection responsibilities. The Nation’s infrastructure protection effort must harness 
the capabilities of the private sector to achieve a prudent level of security without 
hindering productivity, trade, or economic growth... 43 Contextual background is a 
necessary to meet these mandates. 

41 Robert Steele, The New Craft of Intelligence Personal, Public and Political (Oakton, Virginia: OSS 
International Press, 2002), 147-161. 

42 Director of National Intelligence, Report on the Progress of the Director of National Intelligence in 
Implementing the Intelligence Reform and Terrorism Prevention Act of2004, 1, 
http://www.fas.org/irp/dni/implement.html (accessed January 12, 2007). 

43 George W. Bush, National Strategy’for Homeland Security’ (Washington, D.C.: Office of 
Homeland Security, 2002), 5-6. 
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In May, 2006 the Director of National Intelligence (DNI) affirmed the importance 
of using data warehousing to store intelligence contextual background when he reported 
to Congress on current ODNI efforts to establish digital libraries of all new intelligence 
products. He also outlined efforts to improve the ability of analysts to access all relevant 
information for their analysis, both open source and classified, regardless of original 
author. 44 It is critical that analysts have access to this data warehouse (library) of topical 
intelligence products in order to receive the contextual background necessary for them to 
produce contextually accurate intelligence products. It is similarly important for NTR to 
have a data warehouse (library) of contextual background for intelligence products so that 
they may produce better decisions. DHS has funded open source data warehouses such 
as the Memorial Institute for the Prevention of Terrorism (MIPT) Terrorism Knowledge 
Base (TKB) database to address this need. 45 Existing investment in these data 
warehouses can be leveraged, they provide a readily available, government funded, vetted 
source of contextual background that can be fused to intelligence products. Private sector 
assets and government funded, open source contextual background data warehouses are 
two of many valuable assets that must be fully integrated into the intelligence cycle. 

The research to date provides valuable insight into the importance of contextual 
background, the importance of non traditional recipients, and the importance of 
intelligence in decision making. As the provision of unclassified intelligence to NTR is 
relatively new, there is little direct research on the impact of context in this area; this 
impact must be implied from experience with traditional intelligence recipients combined 
with the impact of context in the decision making of NTR in areas other than intelligence. 

5. Technology and Contextual Background 

The literature shows that decision making is impaired when insufficient 
information is available; it also documents that too much information (or information 
overload) impairs decision-making. Technology has the potential to impair decision 
making through the provision of too much information, the “white noise” discussed by 
Lisa Palmieri, or to assist by increasing a decision maker’s ability to acquire, transform 

44 Report on the Progress of the Director of National Intelligence in Implementing the Intelligence 
Reform and Terrorism Prevention Act of2004, 7. 

45 MIPT, About MIPT Terrorism Knowledge Base, Memorial Institute for the Prevention of 
Terrorism, 1, http://www.tkb.org/AboutTKB.jsp (accessed September 3, 2006). 
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and explore knowledge as envisioned by the IC. Technological assistance in providing 
the right amount and type of knowledge to decision makers has the potential to improve 
decisions, increase decision timeliness, and decrease staff support requirements. Given 
the lack of intelligence specialists in NTR disciplines outside of law enforcement, and the 
fact that 79% of police departments have twenty-five or fewer sworn officers, technology 
support is critical if we are to effectively engage NTR in homeland security. 46 

A review of knowledge management literature shows design parallels between 
“expert systems” and the fusion center/Joint Terrorism Task Force process. Fusion 
Centers and Joint Terrorism Task Forces act as knowledge management engines, 
managing the interface between NTR, the national knowledge base, and regional 
knowledge contributed by end users. Software engineers utilize the concept of a “context 
manager” in designing automated software which links disparate sources of data. Some 
believe that, “...the ability to represent and manipulate context will be an extremely 
important part of providing semantic integration in multi-database systems.” 47 

Intelligence products with CBF can function as a critical hub or node in 
networking multiple databases of open source contextual background and NTR; the 
producer (expert) manages the interface between the warehouses and NTR through the 
selection of appropriate hyperlinks. Hyperlinks have already been utilized by some 
unclassified intelligence providers, such as the Illinois State Terrorism Information 
Center (STIC) to provide contextual background. 48 The diagram below outlines the 
impact of such hyperlinks on a small part of the homeland security infonnation network. 


46 Geoffrey Alpert, Police Pursuit and the Use of Force. A Final Report to the National Institute of 
Justice (Washington, D.C.: National Criminal Justice Reference Service, [1997]), 7, 
http://www.ncjrs.gov/pdffiles/164831.pdf (accessed May 25, 2006). 

47 Zhengxin Chen, Computational Intelligence for Decision Support (Boca Raton, FL: CRC Press, 
2000), 27, http://www.engnetbase.com.libproxy.nps.navy.mil/books/482/1799_PDF_TOC.pdf (accessed 
May 26, 2006). 

48 Kevin Eack (Illinois State Police, State Terrorism Intelligence Center), interview by author, 
Springfield, Illinois, June 18, 2006. 
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Figure 3. Homeland security information source network 

Understanding the critical importance of improving the usefulness of intelligence 
products for decision makers, the IC has utilized varied methods to provide user defined 
access to contextual background for intelligence products, including personal briefings by 

intelligence professionals, ongoing feedback to producers on provided intelligence 
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products as part of the intelligence cycle, along with technological solutions such as the 
maintenance of searchable data warehouses of contextual background and a finished 
classified intelligence dissemination system (Intelink). The experience with Intelink 
highlights the importance of technology acceptance by decision makers; Intelink is a 
secure network of intelligence databases that supports user defined receipt of customized 
intelligence products. 49 It is intended to “provide robust and timely access to all 
available intelligence information, regardless of location, medium, or fonnat, for all 
interested users ... who are authorized access.” 50 Incorporating collaborative computing 
on this secure network, recipients are able to teleconference and whiteboard with live 
audio, video, document, and application share capability along with simultaneous access 
to multiple intelligence data warehouses. Intelink was designed not to just “push” 
intelligence information; it allows recipients to “pull” information as well. 51 

Intelink has not been as successful as envisioned. Many intelligence recipients 
have expressed a preference for hard copies of reports, personal briefings and traditional 
communication methods over Intelink. 52 As reflected in Technology Acceptance Model 
research, ease of use and perceived usefulness are significant factors in the successful 
implementation of any solution. Despite the extensive capabilities of Intelink, users 
“claim they go first to the Agency web sites, find no information at all, usually become 
quickly frustrated, and log off with the impression the intelligence agencies do not store 
information on Intelink.” 53 Decision makers want a summary of information the briefer 
believes is important (push) and want additional information the decision maker believes 
may be important from the briefer immediately thereafter (pull). Intelligence system 

49 Lome Teitelbaum, The Impact of the Information Revolution on Policymakers' use of Intelligence 
Analysis (Santa Monica, California: RAND, 2005), 98, 

http://www.rand.org/pubs/rgs_dissertations/2005/RAND_RGSD186.pdf (accessed May 24, 2006). 

50 Fredrick Martin, Top Secret Intranet: How U.S. Intelligence Built Intelink - the Worlds Largest 
most Secure Network (Upper Saddle River, NJ: Prentice Hall, 1999), 1-380. 

51 Patrick Kelly, Intelligence Support to Homeland Security’: Supporting the Supporting Effort 
(Carlisle Barracks, Pennsylvania: U.S. Army War College, 2002), 29, 
https://www.hsdl.org/homesec/docs/dtic/ADA404363.pdf (accessed May 21, 2006). 

52 Teitelbaum, The Impact of the Information Revolution on Policymakers' use of Intelligence 
Analysis, 101. 

53 Ibid., 106. 
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design must support the needs and preferences of decision-makers or run the risk of 
rejection, regardless of how exceptional the technology or the potential of the system. 

6. Technology Acceptance 

The theory of reasoned actions (TRA) serves as a theoretical base for examining 
technology acceptance. 54 TRA posits that an individual’s beliefs influence their attitudes, 
that when combined with societal norms, drives behavioral intentions, leading to actual 
behavior. Based on TRA, the technology acceptance model (TAM) is an established 
method of predicting user acceptance. In this model, perceived “Ease of Use” and 
perceived “Usefulness” explain why individuals accept or do not accept technology. 55 A 
review of previous studies shows that TAM, with strong empirical support, has become a 
dominant model for predicting technology adoption. 56 TAM is one method to predict 
user acceptance before large scale investment or commitment to a technology in mission 
critical systems. 

Perceived ease of use is the degree to which a user believes that using a 
technology would be effortless, and perceived usefulness is the degree to which a user 
believes that a technology would improve perfonnance, production or effectiveness. Of 
the two constructs, it appears that usefulness is critical, “no amount of ease of use can 
compensate for a system that does not perform a useful function.” 57 The use of 
technology in knowledge management systems has been studied using TAM to determine 
factors that impact loyal use; both perceived usefulness and ease of use were found to be 
factors positively related to loyal use. 58 

54 M. Fishbein and I. Ajzen, Belief, Attitude, Intention, and Behavior: An Introduction to Theory’ and 
Research (MA: Addison-Wesley Reading, MA, 1975), 21-52. 

55 F. D. Davis, R. P. Bagozzi and P. R. Warshaw, User Acceptance of Computer Technology’: A 
Comparison of Two Theoretical Models, Management Science 35, no. 8 (1989), 982-1003. 

56 Chienting Lin and others, Examining User Acceptance of COPLINK Technologies by Law 
Enforcement Officers: A Survey Study (Arizona: University of Arizona,2002), 2, 

http://citeseer.ist.psu.edU/cache/papers/cs2/365/http:zSzzSzwww.digitalgovernment.orgzSzlibraryzSzlibrar 
yzSzdgo2002zSz..zSzpdfzSzlin.pdf/examining-user-acceptance-of.pdf (accessed August 20, 2006). 

57 F. D. Davis, R. P. Bagozzi and P. R. Warshaw, “User Acceptance of Computer Technology: A 
Comparison of Two Theoretical Models,” Management Science 35, no. 8 (1989), 333. 

58 Paul Clay, Alan Dennis and Dong-Gil Ko, Factors Affecting the Loyal use of Knowledge 
Management Systems (Indiana: Indiana University,2005), 1-10, 
http://csdl2.computer.org/comp/proceedings/hicss/2005/2268/08/22680251c.pdf. 
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E. RESEARCH QUESTIONS 

• Does CBF increase customer satisfaction “value” for NTR as defined by 
DHS/FBI? 

• Does CBF increase customer satisfaction “quality” for NTR as defined by 
DHS/FBI? 

• Will NTR find hyperlink technology applications in unclassified intelligence as 
“easy to use” as defined by technology acceptance model research? 

• Will NTR perceive hyperlink technology applications in unclassified intelligence 
as “useful” as defined by technology acceptance model research? 

• Given a choice, will NTR believe a CBF intelligence document is of greater value 
to themselves and their organization than a non CBF product? 

F. CHAPTERS 

This research reviews hermeneutic theories, prior studies on the impact of 
contextual background in decision making, current unclassified intelligence distribution, 
and technology acceptance research. The acceptance of hyperlink technology to achieve 
CBF is evaluated utilizing previously validated technology acceptance model factors of 
ease of use and usefulness. The impact of CBF on value and quality factors is 
independently tested using measures defined by DHS and FBI. 

Chapter I serves as an introduction and summary then presents important 
background and context to this research, reviewing relevant literature on the impact of 
context in decision making, technology acceptance, and the direction set by senior policy 
makers in intelligence distribution. The significant citations of senior policy makers, 
researchers and practitioners leads to the conclusion that homeland security decision 
makers require contextual background for intelligence products in order to best use that 
intelligence. 
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Chapter II reviews the existing provision of intelligence products in context with 
an emphasis on unclassified material. Multiple strategies in local, regional, state, federal 
and international agencies are reviewed. 

Chapter III establishes hypothesis’ and outlines the process of building, testing, 
and validating the testing process, leading to the results of that testing. Subject matter 
experts were consulted at every step of the process to ensure both validity and practical 
application. With the input of subject matter experts (SME), the goal of conducting 
valid, scholarly research that improves perfonnance in the GWOT was paramount. The 
implementation of well intentioned, reasonable strategies and tactics is often necessary 
without the benefit of validated research, CHDS provides both the mandate and 
opportunity to research and validate both strategies and tactics. 

The final chapter presents recommendations based on the statements of senior 
policy makers, researchers and practitioners, combined with feedback from NTR 
presented with both traditional and CBF products. 
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II. EXISTING PROVISION OF INTELLIGENCE PRODUCTS IN 

CONTEXT 


The provision of contextual background in intelligence can be described as “the 
paradox of plenty.” Although context is available in many open source Internet 
accessible databases that are relevant to intelligence products distributed to NTR, the 
distributed data warehouses, lack of linking between warehouses and intelligence 
products, lack of reconciliation with intelligence products, along with the sheer volume of 
context interferes with user access to this “plenty .”59 

A RAND study conducted for the FBI summarized the importance of providing 
appropriate intelligence products to NTR. This report indicated that a review of best 
practices of multiple country threat assessments demonstrated a need for products to be: 

• Tailored to meet requests from government departments, police, and 
private industry. 

• Frequently incorporate a process for sending “scrubbed” versions to non- 
cleared customers. 60 

“Tailoring” intelligence for diverse recipients cuts to the heart of NTR need for 
different levels of contextual background. While historically recipients may have been 
relatively homogenous in discipline and hermeneutic understanding of intelligence 
products, the current diversity of disciplines and contextual knowledge of NTR ensures 
that “one size fits all” unclassified intelligence products will fit almost no one. 


A. DEPARTMENT OF THE ARMY 

In Police Intelligence, the United States Military uses hyperlinks to individually 
“tailor” unclassified intelligence products. An example of this networking of open source 
material to unclassified intelligence documents is demonstrated in the July 2006 Police 


59 Nazli Choucri, Stuart Madnick and Michael Siegel, Improving National and Homeland Security 
through Context Knowledge Representation and Reasoning Technologie. (Working Paper, Composite 
Information Systems Laboratory, Sloan School of Management, Massachusetts Institute of Technology, 
Cambridge, MA, 2006), 1-25, http://web.mit.edu/smadnick/www/wp/2006-03.pdf (accessed August 3, 
2006). 

60 Peter Chalk, Seth Jones and William Rosenau, Understanding the Enemy: Lessons for Effective 
Threat Assessments (Santa Monica, CA: RAND, 2004), 9. 
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Intelligence Operations Field Manual. 61 Hyperlinks to open source websites such as FBI, 
news agencies, and the Postal Service are used as examples of fusing unclassified 
intelligence products with open source material in a sample unclassified intelligence 
product. As these hyperlinks are standard, “ease of use” should be same as the 
experimental product in this research. This use of hyperlink technology to allow a “semi¬ 
custom” user defined experience within the confines of open source li nk s provided by the 
analyst is one method to meet the recommendations of RAND, given diverse individual 
recipient needs across a wide geographic area. 

The U.S. Military is not unique in its use of hyperlinks to provide a “semi¬ 
custom” unclassified intelligence product for diverse recipients. Multiple domestic 
intelligence fusion centers have used the same strategy. The U.S. Capital Regional 
Intelligence Center utilizes hyperlinks in footnote citations in For Official Use Only 
(FOUO) Information Bulletins. These footnotes parallel the Military Police Intelligence 
Operations Field Manual, directing interested recipients to open source material, 
primarily news reports. 62 Generally, these hyperlinks are to original source material of 
articles summarized in the report, rather than to contextual background. 


B. NEWS AGENCY 

An example of CBF for a diverse audience can be found at the website of 
television producer “FOX NEWS.” On the FOX NEWS, Homeland Security, Weapons 
of Mass Destruction website, hyperlinks take interested readers to contextual background 
on such keywords as “anthrax,” “dirty bomb,” and “e bomb.” Additionally, a 
“background” box is imbedded in the article with hyperlinks to additional reliable open 
source data warehouses such as “RAND,” “ready.gov,” and “U.S. Army Institute of 
Infectious Disease.”63 This is an example of CBF utilizing accurate, reliable and vetted 

61 Department of the Army, Police Intelligence Operations (Washington, D.C.: Headquarters, 
Department of the Army, 2006), B-l, http://www.fas.org/irp/doddir/army/fm3-19-50.pdf (accessed August 
4, 2006). 

62 National Capital Regional Information Center Information Bulletin IB-06-04, June 30, 2006. 

63 Liza Porteus, Weapons of Mass Destruction Handbook, Fox News, 1, 

http ://www.foxnews.com/story/0,2933,76887,00.html?sPage=fnc.specialsections/homelandsecurity 
(accessed September 10, 2006). 
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government sources for a diverse audience that is examined in this research. The 
“usefulness” of CBF in these products should be similar to the usefulness factor 
examined in this research. 

C. DHS/FBI 

Standard joint FBI/DHS unclassified intelligence products for law enforcement 
are distributed in PDF format, generally without hyperlink or other networking with 
contextual background. These products are distributed to police, fire, EMS, Public 
Health and private sector recipients on a regular basis. 64 In this “one size fits all” 
distribution, the analyst must include contextual background they believe is required by 
the diverse recipients without making the product so long that it will not be read. Each of 
these products contains a customer satisfaction survey that measures the value and quality 
factors utilized in this research. 

Internally, the Department of Homeland Security has the capacity for automated 
contextual linking within the twenty-one agencies in the department through 250,000 
“Autonomy” software seat licenses. According to Autonomy co-founder Richard Gaunt, 
“Autonomy’s intelligence data operating layer (IDOL) provides that context, as well as 
the ability to bring together relevant information from disparate sources.” 65 

The FBI provides both unclassified and classified intelligence to law enforcement 
through 4,000 officers and agents assigned to a network of 101 Joint Terrorism Task 
Forces (JTTF) linked to each FBI field office. 66 Supported by FBI Field Intelligence 


64 Northern California Regional Terrorism Threat Assessment Center, Northern California Regional 
Terrorism Threat Assessment Center, A Local, State and Federal Partnership with the San Francisco FBI, 
FIG, and JTTF (San Francisco; Northern California Regional Terrorism Threat Assessment Center, 2006), 
1 . 


65 Autonomy Corporation pic, Industry Analyst Research Highlights Autonomy's Technology’ Over 
Legacy Rules-Based Only Approach (Sunnyvale, CA: Autonomy, 2006), 1, 
http://www.autonomy.com/content/News/Releases/2002/1104.html (accessed August 6, 2006). 

66 U.S. Department of Justice, Fact Sheet: Department of Justice Anti-Terrorism Efforts since 11 
September 2001, Release No. 202-514-2007 (Washington, D.C., 2006), 1, 
http://www.usdoj.gov/opa/pr/2006/September/06_opa_590.html (accessed January 12, 2007). 
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Groups (FIG), local law enforcement liaisons are provided security clearances and 
encouraged to participate either full time or part time in task force operations, providing a 
layer of geographically tailored intelligence. 


D. FUSION CENTERS 

A growing network of DHS supported state and local fusion centers provide a link 
between intelligence agencies and homeland security professionals, either scrubbing 
classified intelligence, producing independent intelligence, or passing through 
unclassified FBI/DHS products to their geographic area. These centers, with diverse 
leadership structures and discipline participation, represent one method of providing 
“tailored” intelligence that a limited geographic area might need. Just as the military 
hyperlink model provides an individualized semi-custom user defined experience across a 
wide geographic area, the fusion center network makes possible the customization of 
intelligence for a specific geographic region. Some fusion centers, such as the Illinois 
Statewide Terrorism Intelligence Center have utilized hyperlinks to fuse contextual 
background to their intelligence reports in efforts to improve intelligence usefulness. 


E. NEW YORK POLICE DEPARTMENT 

The New York Police Department is unique among the approximately 18,000 
U.S. law enforcement agencies due not only to its experience with terrorism at the World 
Trade Center, but the sheer size of the department, its responsibilities, and its resources. 
Senior Advisor Brian Jenkins from the RAND Corporation summarized that “They’re 
developing best practices here that should be embraced across the country. The Feds 
could leam from them.” 67 Their “NYPD Shield” program is a continuation of New York 
public/private partnership, with the department emailing unclassified intelligence 
products to private sector partners who request participation. The “Shield” intelligence 
products are currently issued as PowerPoint presentations. These presentations include 
significant contextual background including photographs and maps. On the companion 

67 Brian Jenkins (RAND Corporation), Interview with the National Tactical Officers Association, 
February 2005, 58-71, http://www.newyorker.com/fact/content/articles/050725fa_fact2. 
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password protected website, access to contextual background including links to the MIPT 
Terrorism Knowledge Base, Terrorism Timelines, an archive of previous intelligence 
products, and a small library of terrorism security articles is available. These unclassified 
intelligence PowerPoint reports are being secondarily distributed; on August 17, 2006, 
the California Office of Emergency Services redistributed the NYPD Shield Unclassified 
Mumbai PowerPoint to California Fire Chiefs with the comments “Interesting 
information, don't kn ow if you have seen it...the attached PowerPoint presentation is an 
excellent overview of the Mumbai rail attacks of July 11, 2006...” Secondary distribution 
and expert review, while antidotal, indicate superior NYPD product usefulness. 


F. DEPARTMENT OF DEFENSE 

The Department of Defense (DoD) utilizes Verity/Autonomy “K2” and “Profiler” 
software to provide internal context to users of some intelligence systems. This software 
performs multiple contextual background functions. Upon data entry, a contextual 
“frame” is built around the data based on the totality of the data being entered and the 
profile of the submitter; in essence, the software attaches queries that match the data at 
the time of data entry based on the context of the entry. This system, utilizing original 
source data, comes closer to providing a hermeneutic understanding of intelligence; 
intelligence producers are not required to substitute open source material in place of 
classified original sources. Users of the system build a profile of themselves, building 
communities of interest that the software uses to prioritize system inquiry responses. The 
software builds upon that profile based on entry and query history, subsequently using 
this contextual base to not only prioritize information that a user attempts to “pull” from 
the system, but to proactively create reports based on the user’s contextual profile. It 
“pushes” reports based on new data entered into the system that matches the self- 
described and system use contextual profile. Intelligence “pulled” from and “pushed” by 
the system is provided in context based on these “explicit” and “implicit” profiles of 
users. 68 This software appears to be designed primarily for analyst use rather than for 
end users of intelligence. 

68 Christian Scott (Autonomy Software DHS Program Manager), interview by author, Sunnyvale, 
California, August 14, 2006. 
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G. 


INTERPOL 


Interpol, the international organization of 184 member countries, has several 
methods of disseminating terrorism information to member countries including working 
meetings, warning lists, analytical reports, an indices system, “orange” notices, and 
special notices. 69 Interpol does not generally receive classified intelligence, but rather 
intelligence at the Law Enforcement Sensitive level, therefore, there is no need to 
declassify, amend or “scrub” intelligence that is received from members. Any member 
country can tell Interpol who may see information that they provide to Interpol, in 
essence, the originating member country remains the proprietary owner of information. 70 
Interpol’s I 24/7 virtual private network connects all member countries and is used to 
distribute the law enforcement sensitive intelligence. This is a law enforcement only 
system that by policy does not include fire, private sector, public health, or emergency 
management. Information generally stays within law enforcement; however, it is 
possible for member agencies to share outside of law enforcement if there is a 
demonstrated need, with the permission of the originating member. Art theft infonnation 
is one routine case where information is shared outside law enforcement, being routinely 
distributed to art dealers in the private sector in order to recover stolen pieces. 71 The 
Interpol I 24/7 network does not provide hyperlink or other networking to context or 
background information, acting instead as a distribution system for the unclassified 
intelligence products of members. 72 In a manner similar to DHS/FBI PDF files, 
intelligence producers must include contextual background as they feel appropriate. 
Unlike DHS/FBI products, Interpol products are generally distributed to only one 
discipline, law enforcement. 


69 INTERPOL, INTERPOL Notices. (Lyon, Fiance: INTERPOL, 2006), 1-2, 
http://www.interpol.int/Public/ICPO/FactSheets/GI02.pdf (accessed July 27, 2006). 

70 Public Affairs Officer (Public Affairs Office, U.S. Justice Department), interview by author, 
Washington, D.C., July 5, 2006. 

71 Public Affairs Officer (Public Affairs Office, U.S. Justice Department), interview by author, 
Washington, D.C., July 1, 2006. 

72 INTERPOL, Stolen Works of Art (Lyon, France: INTERPOL, 2006). 
http://www.interpol.int/Public/WorkOfArt/Default.asp(accessed August 8, 2006), 1. 
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H. JANE’S INFORMATION GROUP ONLINE 

Jane’s Infonnation Group Online is a subscription based Internet resource that 
describes itself as: 

...a world leading provider of intelligence and analysis on national and 
international defense, security and risk developments. Jane’s is an 
independent organization with an unrivalled reputation of accuracy, 
authority and impartiality. Jane’s delivers partners and clients a strategic 
advantage from intelligence acquired by a unique worldwide network of 
independent analysts. Governments, militaries, business leaders and 
academics in over 180 countries rely on Jane’s providing timely and 
insightful information on threat and security issues. 

Founded in 1898, Jane’s has always been at the forefront of publishing 
and developing infonnation solutions for its customers. Jane’s provides a 
unique and independent service that allows its clients the ability to make 
mission or business critical decisions with assurance. Jane’s understands 
the importance of accurate infonnation to navigate business and 
governmental leaders through an increasingly turbulent world. Every day 
crucial decisions are made by Jane’s customers often based purely on 
Jane’s infonnation. 73 

In a manner similar to Fox News, Jane’s utilizes CBF through the use of 
hyperlinks to both proprietary and open source government material in their intelligence 
analysis. In addition to internal links, they utilize a “compendium of links to government 
information sites, counter-terrorism legislation, reports and position papers... and best 
practice guidance.. .” 74 In this research the type of CBF utilized by Fox News and Jane’s 
is evaluated for ease of use, usefulness, value and quality for application to unclassified 
intelligence distribution to NTR. 


I. CONCLUSIONS 

Because the practice of providing unclassified intelligence to NTR is relatively 
new, IC provision of contextual background trails private sector “smart practices.” 
Private sector producers such as Fox News and Jane’s have significant experience in 
producing reports for a broad and diverse audience, while the IC has traditionally 

73 Jane’s Online, Jane’s Intelligence and insight you can trust (Alexandria, Virginia: Jane’s 
Information Group, 2005), 1, http://www.janes.com/company/about/ (accessed 10/11/2006). 

74 Ibid, 2. 
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supplied classified intelligence for a relatively homogenous audience. The geographic, 
jurisdictional, and disciplinary diversity of NTR resembles the diverse audience of 
private sector producers such as Jane’s. This study explores the impact of CBF used by 
the private sector on unclassified intelligence distributed to NTR. Given the similarly 
between the diverse audiences of private sector intelligence producers and NTR, research 
into other smart practices of the private sector is warranted; the IC has significant 
experience in intelligence analysis, while the private sector has significant experience in 
communicating with diverse audiences with multiple interests, both of which can be 
exploited in improving intelligence communication with NTR. 

Contextual background has been established as a critical factor in decision 
making from the time of Plato through the current global war on terror. The Department 
of Homeland Security and Federal Bureau of Investigation have established value and 
quality as two factors that reflect customer satisfaction with their unclassified intelligence 
products, providing an established, standardized method of measuring the impact of CBF 
on those products. Technology acceptance model factors of usefulness and ease of use 
also provide valuable feedback on the impact of CBF. This previously validated TAM 
research provides an established, standardized method of measuring whether the 
hyperlinks utilized to achieve CBF will be accepted in widespread application. Any 
changes to unclassified intelligence products that are rated highly for usefulness, ease of 
use, value and quality that will be widely accepted while improving decision making and 
customer satisfaction should be considered for immediate application. 
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III. HYPOTHESIS TESTING 


A. HYPOTHESIS’ 

A null hypothesis (H 0 ) is a hypothesis that can be statistically examined. It is 
presumed to be true until statistical analysis demonstrates it to be false, or nullified, in 
which case the alternative hypothesis (Hi) may be accepted. A Null hypothesis can be 
designed to test that there is no difference between variables, it is then evaluated and the 
results examined to determine what the probability is that observed differences between 
variables are by chance. In determining whether to reject the null hypothesis in favor of 
the alternative hypothesis, we must detennine an appropriate level of significance that 
must be met to reject the null hypothesis. The smaller this p-value is, the more 
significant the result; 5% or .05 is generally accepted as significant, while 1% or .01 is 
more statistically powerful, as it is a much more difficult threshold to meet. In this 
research the following two null hypotheses are evaluated to determine whether to accept 
the alternative hypothesis: 


Ho: There is no difference in intelligence product perceived value as represented by 
DHS/FBI customer satisfaction surveys, on average, with the application of CBF. 

Hi: An intelligence product with hyperlinks to open source contextual background is 
perceived as more valuable based on DHS/FBI customer satisfaction surveys than a non- 
CBF product, on average. 


Ho: There is no difference in intelligence product perceived quality as represented by 
DHS/FBI customer satisfaction surveys, on average, with the application of CBF. 

Hp An intelligence product with hyperlinks to open source contextual background is 
perceived as higher quality based on DHS/FBI customer satisfaction surveys than a non- 
CBF product, on average. 
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B. METHODOLOGY 

In this empirical research, a national survey for homeland security professionals 
was designed to discover the impact of hyperlink CBF on unclassified intelligence 
products. A control product was presented and a base measurement of DHS/FBI quality 
and value factors was established. The experimental CBF product was presented and a 
second measurement of DHS/FBI quality and value factors was recorded, establishing the 
positive or negative impact of CBF on the customer satisfaction factors established by 
DHS/FBI. Questions from previously validated technology acceptance model factors of 
ease of use and usefulness were then presented in order to determine if intelligence 
recipients would accept hyperlinks to achieve CBF. A final forced choice question to 
determine preference for control or experimental products was presented to confirm 
DHS/FBI factor results. 

1. Instrument Construction 

Two identical sample intelligence products were produced for the survey, one text 
file as similar as practical to a typical intelligence product distributed to non traditional 
recipients and a second identical product fused with hyperlinks to contextual background 
(CBF) found at reliable, government funded websites documented in Appendix A. The 
material in the sample report was unclassified, having been adapted from the Southern 
Poverty Law Center Intelligence Project Eco-Radicalism, open source, online report, 
allowing this research to remain unclassified. 75 The intelligence sample product was 
distributed to four intelligence subject matter experts from local, county, state, and 
federal jurisdictions and based on their feedback the sample was shortened. 

The survey was prepared for “Zoomerang” online distribution. The survey 
presented a standard intelligence product, asked seven questions, presented a CBF 
intelligence product, requesting a response to the same seven questions along with twelve 
additional questions directly related to ease of use and usefulness consistent with the 
Technology Acceptance Model (TAM). The first seven survey questions after both 
standard and CBF versions mirror the DHS/FBI customer satisfaction survey with the 

75 Southern Poverty Law Center, Eco-Radicalism, Feds Across Country’ to Smash Eco-Terrorists ’ 
(Montgomery, Alabama: Southern Poverty Law Center, 2005), 1, 

http://www.splcenter.org/intel/intelreport/article.jsp7aicH608 (accessed September 22, 2006). 
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questions divided into perceived value and perceived quality factors; Appendix D 
contains a copy of this two factor customer satisfaction survey that is currently 
distributed with DHS/FBI unclassified intelligence products. These questions provide 
direct feedback to DHS/FBI on the impact of CBF on the measures that they seek from 
their “customers.” These questions utilized the same five-point Likert scale as DHS/FBI 
products, with Strongly Disagree on one end and Strongly Agree on the other end, along 
with an N/A option. One question from the DHS/FBI Customer Satisfaction Survey 
regarding timeliness was not utilized, “The product was delivered within established 
guidelines,” as the sample product was not being delivered based on guidelines. All other 
customer satisfaction survey questions and Likert scale are exactly as found in DHS/FBI 
documents. 

Twelve additional questions were adapted from Technology Acceptance Model 
research first validated in by Davis in 1989. TAM was developed to explain and predict 
computer use. 76 In this model, the two factors of “Usefulness” and “Ease of Use” have 
been demonstrated to predict the use of technology. The questions in these factors 
utilized an ordinal Likert 6-point scale with Strongly Disagree at one extreme and 
Strongly Agree at the other end. A final question in the survey asked respondents to 
directly identify whether a CBF or standard product would be of greater value to their 
organization. 

The pilot study was sent to fifteen subject matter experts (SME) in public health, 
fire, emergency management, and law enforcement for feedback. The control sample 
intelligence document was designed consistent with the model below, with the researcher 
producing, distributing and receiving feedback on the sample intelligence document in a 
manner similar to existing intelligence production. 


76 User Acceptance of Computer Technology’: A Comparison of Two Theoretical Models, 982-1003. 
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The experimental sample intelligence document with hyperlinks was designed 
consistent with the model below, with the researcher producing, distributing and 
receiving feedback. 
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The hyperlinks utilized in the sample survey product led to contextual background 
at government funded open source Internet sites vetted by the researcher (Appendix A). 
In addition, one hyperlink to a company website specifically identified in the intelligence 
document was included, http://www.huntingdon.com. 

SME suggested a “header” and “legalistic sounding disclaimer” to make the 
survey look more realistic, as well as changing one jurisdiction choice from “city” to 
“local.” After making these adjustments, the survey was distributed to the target 

audience, who were requested to complete the survey and forward the survey to other 
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homeland security professionals. As this research utilizes customer satisfaction survey 
questions from the Department of Homeland Security and Federal Bureau of 
Investigation, two major producers of unclassified intelligence, a bifurcated sampling 
method was utilized. In one frame invitations to participate in the survey were emailed to 
current and former students of the DHS sponsored Naval Postgraduate School, Center for 
Homeland Defense and Security (CHDS). These students, all of which were selected by 
DHS to attend the school, include diverse homeland security disciplines including, police, 
fire, public health, military, EMS, and emergency management. This sampling frame is 
intended to represent intelligence recipients of interest to DHS. As every current and 
former student was invited to participate, these students represent the entire universe of 
CHDS students and no further sampling was needed to survey this population. 

A second sampling frame consisted of former students of the FBI National 
Academy (FBINA). Similar to the DHS sponsored CHDS; each student attending the 
FBINA is sponsored and selected by one of the fifty-six FBI Field Offices. Each student 
attending the FBINA is a member of law enforcement, representing local, county, state 
and international jurisdictions. By 2005, over 38,000 students from 220 sessions had 
attended FBINA, so further sampling was required. Several domestic students from 
FBINA session 214 were selected to represent intelligence recipients of interest to the 
FBI; several CHDS law enforcement survey recipients have previously attended FBINA, 
however none had attended session 214, avoiding an overlap of samples. A total of 172 
email invitations to participate in the survey were sent to current and fonner homeland 
security professionals sponsored by the FBI or DHS to attend these courses. Snowball 
style secondary distribution by these initial recipients was encouraged. 

a. DHS/FBI Customer Satisfaction Survey Questions 

Quality: 

1. The Product was timely and relevant to your mission, programs, priorities, or 
initiatives. 

2. The product was clear and logical in the presentation of information with 
supported judgments and conclusions. 

3. The product is reliable; i.e., sources well documented and reputable. 
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Value: 

1. The product would contribute to satisfying intelligence gaps or predicating 
cases or intelligence operations, especially previously unknown areas. 

2. The product would result in a change in investigative or intelligence priorities 
and/or a shift from unaddressed to addressed work, or vice versa. 

3. The product would result in more informed decisions concerning investigative 
or intelligence initiatives and/or resource allocation. 

4. The product would identify new information associated with pending matters 
or offered insights into information that could change the working premise in a 
program or initiative. 

Figure 1 is a screen shot of a DHS/FBI customer satisfaction survey 
distributed to NTR. 


b. Technology Acceptance Model 

Perceived usefulness: 

1. Using the intelligence product with hyperlinks in my job would enable me to 
accomplish tasks more quickly. 

2. Using intelligence products with hyperlinks would improve my job 
performance. 

3. Using intelligence products with hyperlinks would increase my productivity. 

4. Using intelligence products with hyperlinks would enhance my effectiveness 

on the job. 

5. Using intelligence products with hyperlinks would make it easier to do my job 

6. I would find intelligence products with hyperlinks useful in my job. 

Perceived ease of use: 

1. Learning to utilize intelligence products with hyperlinks would be easy for me. 

2. I would find it easy to use intelligence hyperlinks to obtain decision making 
information. 

3. My interaction with the intelligence product with hyperlinks was clear and 
understandable. 

4. I found the intelligence hyperlinks to be flexible to interact with. 

5. It would be easy for me to become skillful at using intelligence products with 

hyperlinks. 

6. I found the intelligence product with hyperlinks easy to use. 
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c. Preference 

1. Which type of intelligence product would be of greater value to you and your 

organization? 

The “Zoomerang” online survey tool utilized is included as Appendix E. 
The data obtained from the online survey was downloaded into a Statistical Package for 
the Social Sciences (SPSS) program for descriptive, bivariate, multivariate, correlation, 
regression and reliability analysis. 


C. LIMITATIONS OF RESEARCH 

It should be noted that the original list of survey recipients, although 
geographically diverse, was based on two advanced educational programs, which may 
introduce bias in the results; these professionals had attended the FBI National Academy 
or the Naval Postgraduate School Center for Homeland Defense and Security. As DHS 
and the FBI jointly distribute unclassified intelligence products and the questions utilized 
in the survey were in part based on their joint customer satisfaction survey, it is believed 
that sending the survey to students of a DHS sponsored (Center for Homeland Defense 
and Security), and FBI sponsored (FBI National Academy) program would represent a 
reasonable target audience for DHS/FBI intelligence products. Original distribution to 
homeland security students and alumni of the Naval Postgraduate School Center for 
Homeland Defense and Security (NPS CHDS) and FBI National Academy (FBINA) 
graduates may not adequately represent the technological diversity of homeland security 
professionals, given the online fonnat of NPS CHDS and leadership positions of FBINA 
graduates. Pilot study SME feedback, participant free form comments and demographic 
results reflect stronger ecological validity than external validity; while the survey was 
originally widely distributed, the anonymous nature of feedback and low frequency of 
individual jurisdiction/discipline responses demands caution interpreting individual 
discipline and jurisdiction results. 

As research surveys were completed through an anonymous online survey 
instrument, participants were not able to be contacted for response clarification. The 
“snowball” distribution style, where those who were originally requested to participate 
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were encouraged to redistribute to their homeland security contacts, reduces validity, 
while increasing sample size. This distribution style can introduce bias as it reduces the 
likelihood that the respondents represent an appropriate population sample. Secondary 
distribution by these recipients, not controlled by the researcher, was intended to 
similarly represent the target audience of the DHS and the FBI; the extent to which this 
sample does not represent the target audience of these agencies represents potential bias 
in the survey results. 77 

As the survey responses were anonymous, recipients who visited the site or only 
partially completed the survey, then came back to complete the survey, registered as a 
visit, a partial and a complete response. Since one recipient may be included in the 
number of times the site was “visited,” the number of “partial” responses and the number 
of “complete” responses, only completed survey data were utilized for research, and the 
resulting visit/partial/complete response rates are not utilized. 285 surveys were 
completed, 182 partial responses were not utilized, and the website was visited a total of 
648 times. From this data, the response rate may have been anywhere from 44% to 100%. 
Not knowing the true response rate is a limitation in this research. 

The sample was derived from DHS and FBI sponsored educational programs that 
do not generally include private sector participants; one result is that the two million 
private sector security professionals are under represented in this research. 

D. RESULTS 

1. Demographics 

The sample contained 285 responses, all of which were usable. Six discipline 
choices were offered in the survey; law enforcement was the largest single represented 
discipline with 35.1% of responses. Forty-nine “Other” responses fell into the following 
generalized categories: 

United States Coast Guard 

Military 

77 Survey Sampling Methods, Stat Pac, 1, http://www.statpac.com/surveys/sampling.htm (accessed 
September 10, 2006). 
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Homeland Security 

Intelligence 

Public Works 

Medical 

Legal 

Private Sector 


Discipline 



Frequency 

Percent 

Valid Percent 

Cumulative 

Percent 

Valid Other 

49 

17.2 

17.2 


Public Health 

42 

14.7 

14.7 

31.9 

Law Enforcement 

100 

35.1 

35.1 

67.0 

Fire 

51 

17.9 

17.9 

84.9 

EMS 

7 

2.5 

2.5 

87.4 

Emergency Management 

36 

12.6 

12.6 

100.0 

Total 

285 

100.0 

100.0 



Table 1. Discipline Demographics 


Six jurisdictions were represented; the largest single jurisdiction was local 
respondents at 41.1%. Twenty-four “Other” responses fell into the following general 
categories: 

District of Columbia 

Academia 

Tribal 

Multi County 
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Jurisdiction 



Frequency 

Percent 

Valid Percent 

Cumulative 

Percent 

Valid Other 

24 

8.4 

8.4 

8.4 

Federal 

56 

19.6 

19.6 

28.1 

State 

46 

16.1 

16.1 

44.2 

County 

39 

13.7 

13.7 

57.9 

Local 

117 

41.1 

41.1 

98.9 

Private Sector 

3 

1.1 

1.1 

100.0 

Total 

285 

100.0 

100.0 


Table 2. Jurisd 

liction Demographics. 


The largest single discipline/jurisdiction respondent combination was local law 
enforcement, with fifty individual responses or 17.5% of total respondents, followed by 
local fire at 14.4%. Table 2 documents the counts and percentages of each combination 
of jurisdiction and discipline. 


2. Responses DHS/FBI Quality 

The three questions comprising the DHS/FBI Quality component of customer 
satisfaction surveys, 

1. The Product was timely and relevant to your mission, programs, priorities, or 
initiatives. 

2. The product was clear and logical in the presentation of information with 
supported judgments and conclusions. 

3. The product is reliable; i.e., sources well documented and reputable, 
were compared for the standard and CBF products. 
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Question 1 

The Product was timely and relevant to your mission, programs, priorities, or 
initiatives. 


Mean 


Discipline 

Jurisdiction 

Standard 
Timely and 
Relevant 

CBF 

Timely and 
Relevant 

Other 

Other 

3.00 

3.50 

Federal 

2.74 

3.05 

State 

4.18 

4.55 

County 

2.00 

4.00 

Local 

3.29 

3.43 

Private Sector 

3.33 

3.67 

Total 

3.22 

3.58 

Public Health 

Other 

1.88 

2.00 

Federal 

2.00 

2.00 

State 

3.30 

3.33 

County 

3.60 

3.47 

Local 

3.83 

3.67 

Total 

3.15 

3.10 

Law Enforcement 

Other 

3.25 

3.75 

Federal 

3.33 

3.52 

State 

4.14 

4.00 

County 

3.83 

3.83 

Local 

3.57 

3.96 

Total 

3.60 

3.85 

Fire 

Other 

3.00 

3.00 

State 

2.00 

2.00 

County 

3.14 

3.71 

Local 

3.63 

4.00 

Total 

3.51 

3.88 

EMS 

County 

4.00 

4.00 

Local 

3.67 

3.67 

Total 

3.71 

3.71 

Emergency Management 

Other 

3.25 

3.75 

Federal 

3.13 

3.25 

State 

3.38 

3.50 

County 

3.11 

3.22 

Local 

3.86 

4.29 


Total 

3.33 

3.56 

Total 

Other 

2.68 

3.00 

Federal 

3.04 

3.25 

State 

3.77 

3.86 

County 

3.41 

3.54 

Local 

3.61 

3.93 

Private Sector 

3.33 

3.67 


Total 

3.42 

3.66 


Table 3. DHS/FBI “Quality” question 1 response demographics. 
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State law enforcement, local and county public health responses reflect a belief 
that the standard product was more timely and relevant to their mission, programs, 
priorities, or initiatives; federal public health, county law enforcement, state fire, other 
fire, and all EMS respondents did not have a preference in which product was more 
timely and relevant to their mission, programs, priorities while all eighteen others 
preferred the CBF product. On average, public health and federal respondents rated both 
standard and CBF products lower than other disciplines and jurisdictions. In no case did 
federal respondents rate the CBF product lower than the standard product; the mean 
improvement of federal respondents was 5.82%. As a discipline, public health rated the 
CBF slightly lower than the standard product (1.59%). Overall, the mean results for all 
disciplines combined improved 7% with the application of CBF from 3.42 to 3.66 on the 
6-point scale. 
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Question 2 

The product was clear and logical in the presentation of information with 
supported judgments and conclusions. 


Mean 


Discipline 

Jurisdiction 

Standard Clear 
and Logical 

CBF Clear and 
Logical 

Other 

Other 

3.25 

4.40 

Federal 

3.45 

3.81 

State 

3.75 

4.25 

County 

2.00 

5.00 

Local 

3.57 

4.14 

Private Sector 

3.33 

4.00 

Total 

3.49 

4.06 

Public Health 

Other 

2.67 

3.44 

Federal 

2.50 

2.50 

State 

3.50 

4.00 

County 

3.53 

3.93 

Local 

3.83 

4.17 

Total 

3.33 

3.80 

Law Enforcement 

Other 

4.00 

4.25 

Federal 

3.50 

4.13 

State 

3.80 

4.27 

County 

3.33 

4.33 

Local 

3.70 

4.12 

Total 

3.66 

4.16 

Fire 

Other 

3.50 

4.00 

State 

3.00 

3.00 

County 

3.00 

4.00 

Local 

4.02 

4.20 

Total 

3.84 

4.14 

EMS 

County 

4.00 

4.00 

Local 

3.33 

4.17 

Total 

3.43 

4.14 

Emergency Management 

Other 

3.25 

3.75 

Federal 

3.63 

3.75 

State 

3.38 

3.75 

County 

3.33 

3.56 

Local 

4.00 

4.57 


Total 

3.53 

3.86 

Total 

Other 

3.17 

3.88 

Federal 

3.46 

3.89 

State 

3.63 

4.09 

County 

3.33 

3.95 

Local 

3.81 

4.18 

Private Sector 

3.33 

4.00 


Total 

3.59 

4.05 


Table 4. DHS/FBI “Quality” question 2 response demographics. 
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No identified group indicated that the standard product was more clear or logical 
in the presentation of information with supported judgments and conclusions, while 
county EMS, state fire, and federal public health showed no preference. Twenty four 
other combinations of jurisdiction and discipline believed the CBF product was more 
clear and logical. Mean results improved approximately 13% with the application of 
CBF from 3.59 to 4.05 on the 6-point scale. Although never rating the standard product 
higher than the CBF product, public health and federal respondents continued to rate both 
products lower than the average of all disciplines and jurisdictions. 
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Question 3 

The product is reliable; i.e., sources well documented and reputable. 

Mean 


Discipline 

Jurisdiction 

Standard 

Reliable 

CBF Reliable 

Other 

Other 

3.40 

4.20 

Federal 

3.50 

4.33 

State 

3.82 

4.42 

County 

4.00 

5.00 

Local 

3.00 

4.71 

Private Sector 

3.67 

4.67 

Total 

3.51 

4.43 

Public Health 

Other 

2.50 

3.67 

Federal 

2.50 

3.50 

State 

3.11 

4.13 

County 

3.27 

3.87 

Local 

2.83 

4.00 

Total 

2.98 

3.88 

Law Enforcement 

Other 

3.75 

4.75 

Federal 

3.13 

4.17 

State 

3.33 

4.47 

County 

3.83 

4.33 

Local 

3.39 

4.37 

Total 

3.36 

4.35 

Fire 

Other 

3.00 

3.50 

State 

2.00 

2.00 

County 

2.71 

4.29 

Local 

3.78 

4.23 

Total 

3.57 

4.16 

EMS 

Local 

3.17 

4.17 

Total 

3.17 

4.17 

Emergency Management 

Other 

2.50 

4.50 

Federal 

3.38 

3.88 

State 

2.75 

4.50 

County 

3.00 

3.44 


Local 

3.43 

4.50 


Total 

3.06 

4.09 

Total 

Other 

2.96 

4.08 

Federal 

3.28 

4.16 

State 

3.27 

4.34 

County 

3.21 

3.95 

Local 

3.47 

4.32 

Private Sector 

3.67 

4.67 


Total 

3.32 

4.22 


Table 5. DHS/FBI “Quality” question 3 response demographics. 
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No identified group believed the standard product was more reliable; i.e., sources 
well documented and reputable; state fire showed no preference and twenty five other 
combinations of jurisdiction and discipline believed the CBF product was more reliable. 
Mean results improved approximately 27% with the application of CBF from 3.32 to 4.22 
on the 6-point scale, the greatest improvement of any variable reviewed. Public health 
and federal respondents once again rated both products lower than the overall average of 
all disciplines and jurisdictions. 


3. DHS/FBI Quality 

Appendix D reflects the intention of DHS/FBI to define and obtain feedback on 
intelligence product quality by requesting responses to the above three questions under 
the heading of “quality.” The individual responses to the three questions were combined 
to form a DHS/FBI Quality factor; the mean of the three quality questions was calculated 
for both standard and CBF responses by jurisdiction and discipline: 
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Mean 


Discipline 

Jurisdiction 

Standard 

Average 

Quality 

CBF Average 
Quality 

Other 

Other 

3.0833 

3.9167 

Federal 

3.2833 

3.7500 

State 

3.9091 

4.4444 

County 

2.6667 

4.6667 

Local 

3.2857 

4.0952 

Private Sector 

3.4444 

4.1111 

Total 

3.4130 

4.0355 

Public Health 

Other 

2.6296 

3.1852 

Federal 

2.3333 

2.6667 

State 

3.4000 

3.8889 

County 

3.4667 

3.7556 

Local 

3.5000 

3.9444 

Total 

3.2222 

3.6341 

Law Enforcement 

Other 

3.6667 

4.2500 

Federal 

3.4267 

3.9130 

State 

3.8000 

4.2444 

County 

3.6667 

4.1667 

Local 

3.5867 

4.1600 

Total 

3.5867 

4.1190 

Fire 

Other 

3.1667 

3.5000 

State 

2.3333 

2.3333 

County 

2.9524 

4.0000 

Local 

3.8130 

4.1707 

Total 

3.6405 

4.0850 

EMS 

County 

4.6667 

4.6667 

Local 

3.3889 

4.0000 

Total 

3.5714 

4.0952 

Emergency Management 

Other 

3.0000 

4.0000 

Federal 

3.3750 

3.6250 

State 

3.1667 

3.9167 

County 

3.1481 

3.4074 

Local 

3.7619 

4.5238 


Total 

3.3056 

3.8519 

Total 

Other 

3.0000 

3.6667 

Federal 

3.3273 

3.7610 

State 

3.5926 

4.1259 

County 

3.3419 

3.8291 

Local 

3.6439 

4.1624 

Private Sector 

3.4444 

4.1111 


Total 

3.4775 

3.9929 


Table 6. DHS/FBI “Quality” factor response demographics. 
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When the three questions comprising the DHS/FBI quality factor are combined, 
county EMS and state fire showed no preference, while all twenty-five other 
combinations of jurisdiction and discipline indicated that the CBF product was of higher 
quality than the standard product. County EMS rated both products very high (4.66), 
while state fire rated both products very low (2.33). Mean results improved 14.82% with 
the application of CBF from 3.48 to 3.99 on the 6-point scale; overall public health and 
federal respondents followed the same CBF improvement results as all disciplines while 
rating both standard and CBF products lower than other disciplines/jurisdictions. 


4. DHS/FBI Value 

The four questions comprising the DHS/FBI Quality factor of customer 
satisfaction surveys, 

1. The product would contribute to satisfying intelligence gaps or predicating 
cases or intelligence operations, especially previously unknown areas. 

2. The product would result in a change in investigative or intelligence priorities 
and/or a shift from unaddressed to addressed work, or vice versa. 

3. The product would result in more informed decisions concerning investigative 
or intelligence initiatives and/or resource allocation. 

4. The product would identify new information associated with pending matters 
or offered insights into information that could change the working premise in a 
program or initiative. 

were compared for the standard and CBF products. 
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Question 1 

The product would contribute to satisfying intelligence gaps or predicating cases or 
intelligence operations, especially previously u nkn own areas. 


Mean 


Discipline 

Jurisdiction 

Standard 

Satisfying 

Gaps 

CBF Satisfying 
Gaps 

Other 

Other 

3.20 

4.00 

Federal 

2.95 

3.57 

State 

3.50 

4.00 

County 

4.00 

4.00 

Local 

3.29 

3.83 

Private Sector 

3.67 

4.33 

Total 

3.23 

3.81 

Public Health 

Other 

2.50 

3.50 

Federal 

3.00 

3.00 

State 

3.20 

3.38 

County 

3.33 

3.80 

Local 

3.50 

4.17 

Total 

3.15 

3.67 

Law Enforcement 

Other 

3.25 

4.75 

Federal 

3.21 

3.83 

State 

3.20 

4.13 

County 

3.33 

4.33 

Local 

3.26 

4.00 

Total 

3.24 

4.03 

Fire 

Other 

3.00 

3.50 

State 

3.00 

3.00 

County 

3.00 

4.00 

Local 

3.78 

4.03 

Total 

3.63 

3.98 

EMS 

County 

4.00 

3.00 

Local 

3.17 

3.83 

Total 

3.29 

3.71 

Emergency Management 

Other 

3.75 

4.25 

Federal 

3.00 

3.38 

State 

3.25 

4.00 

County 

2.78 

3.33 

Local 

3.29 

4.29 


Total 

3.14 

3.78 

Total 

Other 

3.04 

3.96 

Federal 

3.07 

3.64 

State 

3.28 

3.91 

County 

3.18 

3.79 

Local 

3.45 

4.02 

Private Sector 

3.67 

4.33 


Total 

3.28 

3.89 


Table 7. DHS/FBI “Value” question 1 response demographics. 
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Once again, federal responses paralleled those of other disciplines albeit with 
lower ratings; county EMS respondents preferred the standard product, state fire, federal 
public health, and county “other” showed no preference, and the twenty-three other 
jurisdiction/discipline combinations indicated that the CBF product would better 
contribute to satisfying intelligence gaps or predicating cases or intelligence operations, 
especially previously unknown areas. Mean results improved 18.6% with the application 
of CBF from 3.28 to 3.89 on the 6-point scale. 
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Question 2 

The product would result in a change in investigative or intelligence priorities 
and/or a shift from unaddressed to addressed work, or vice versa. 


Mean 


Discipline 

Jurisdiction 

Standard 

Change 

CBF Change 

Other 

Other 

3.00 

3.40 

Federal 

2.74 

3.30 

State 

3.00 

4.00 

County 

2.00 

5.00 

Local 

3.17 

3.67 

Private Sector 

3.67 

4.00 

Total 

2.93 

3.62 

Public Health 

Other 

2.71 

3.50 

Federal 

3.00 

3.00 

State 

2.25 

3.29 

County 

3.00 

3.67 

Local 

3.50 

3.83 

Total 

2.87 

3.55 

Law Enforcement 

Other 

3.50 

4.00 

Federal 

3.04 

3.67 

State 

3.27 

3.80 

County 

3.17 

3.83 

Local 

3.18 

3.66 

Total 

3.17 

3.71 

Fire 

Other 

2.00 

2.50 

State 

3.00 

2.00 

County 

2.71 

3.57 

Local 

3.19 

3.73 

Total 

3.06 

3.62 

EMS 

County 

4.00 

3.00 

Local 

3.50 

3.67 

Total 

3.57 

3.57 

Emergency Management 

Other 

3.25 

4.00 

Federal 

2.75 

2.88 

State 

3.14 

3.71 

County 

3.11 

3.44 

Local 

3.57 

4.14 


Total 

3.14 

3.57 

Total 

Other 

2.95 

3.57 

Federal 

2.89 

3.39 

State 

2.98 

3.71 

County 

3.00 

3.64 

Local 

3.24 

3.72 

Private Sector 

3.67 

4.00 


Total 

3.08 

3.63 


Table 8. DHS/FBI “Value” question 2 response demographics. 
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County EMS and state fire responses indicate that the standard product would 
result in a change in investigative or intelligence priorities and/or a shift from 
unaddressed to addressed work, or vice versa, federal public health showed no 
preference, while twenty-four other jurisdiction/disciplines preferred the CBF product. 
Mean results improved almost 18% with the application of CBF from 3.08 to 3.63 on the 
6-point scale with federal respondents continuing their tendency to rate all products lower 
than other disciplines. 
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Question 3 

The product would result in more informed decisions concerning investigative or 
intelligence initiatives and/or resource allocation. 


Mean 


Discipline 

Jurisdiction 

Standard 

Informed 

CBF Informed 

Other 

Other 

3.40 

4.20 

Federal 

3.15 

3.76 

State 

3.42 

4.08 

County 

4.00 

5.00 

Local 

2.86 

3.43 

Private Sector 

3.67 

4.33 

Total 

3.25 

3.90 

Public Health 

Other 

3.00 

3.63 

Federal 

3.00 

3.50 

State 

2.67 

3.75 

County 

3.80 

4.07 

Local 

4.00 

4.00 

Total 

3.38 

3.87 

Law Enforcement 

Other 

3.50 

4.50 

Federal 

3.29 

3.83 

State 

3.47 

4.07 

County 

3.33 

4.17 

Local 

3.42 

4.04 

Total 

3.39 

4.02 

Fire 

Other 

3.00 

2.50 

State 

3.00 

3.00 

County 

3.29 

4.14 

Local 

3.62 

3.95 

Total 

3.53 

3.90 

EMS 

County 

3.00 

3.00 

Local 

3.17 

4.17 

Total 

3.14 

4.00 

Emergency Management 

Other 

3.75 

4.00 

Federal 

2.88 

3.25 

State 

3.14 

4.13 

County 

3.33 

3.44 

Local 

4.00 

4.57 


Total 

3.37 

3.83 

Total 

Other 

3.30 

3.87 

Federal 

3.17 

3.71 

State 

3.23 

4.00 

County 

3.51 

3.95 

Local 

3.50 

4.01 

Private Sector 

3.67 

4.33 


Total 

3.38 

3.93 


Table 9. DHS/FBI “Value” question 3 response demographics. 
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“Other” fire indicated that the standard product would result in more informed 
decisions concerning investigative or intelligence initiatives and/or resource allocation, 
county EMS, state fire, and local public health showed no preference and twenty-three 
other combinations indicated that the CBF product was superior with federal respondents 
continuing to rate both products lower than other jurisdictions. Mean results improved 
approximately 16% with the application of CBF from 3.38 to 3.93 on the 6-point scale. 
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Question 4 

The product would identify new information associated with pending matters or 
offered insights into information that could change the working premise in a 
program or initiative. 


Mean 


Discipline 

Jurisdiction 

Standard New 
Information 

CBF New 
Information 

Other 

Other 

3.40 

4.20 

Federal 

3.05 

3.71 

State 

3.75 

4.17 

County 

4.00 

5.00 

Local 

3.33 

3.83 

Private Sector 

4.00 

4.33 

Total 

3.38 

3.96 

Public Health 

Other 

2.88 

3.63 

Federal 

3.00 

3.50 

State 

2.89 

3.50 

County 

3.80 

3.80 

Local 

3.50 

4.17 

Total 

3.33 

3.74 

Law Enforcement 

Other 

3.75 

4.25 

Federal 

3.42 

3.96 

State 

3.40 

4.07 

County 

3.33 

4.00 

Local 

3.34 

3.96 

Total 

3.38 

3.99 

Fire 

Other 

3.00 

2.50 

State 

4.00 

4.00 

County 

3.17 

4.00 

Local 

3.63 

4.03 

Total 

3.55 

3.96 

EMS 

County 

3.00 

4.00 

Local 

3.00 

4.33 

Total 

3.00 

4.29 

Emergency Management 

Other 

3.75 

3.75 

Federal 

3.00 

3.38 

State 

3.50 

3.88 

County 

3.11 

3.44 

Local 

3.57 

4.29 


Total 

3.33 

3.72 

Total 

Other 

3.30 

3.78 

Federal 

3.20 

3.76 

State 

3.42 

3.95 

County 

3.45 

3.82 

Local 

3.44 

4.03 

Private Sector 

4.00 

4.33 


Total 

3.39 

3.92 


Table 10. DHS/FBI “Value” question 4 response demographics. 
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“Other” fire indicated that the standard product would best identify new 
information associated with pending matters or offered insights into information that 
could change the working premise in a program or initiative, “other” emergency 
management, state fire, and county public health showed no preference, all twenty-three 
other combinations preferred the CBF product. Mean results improved over 15% with 
the application of CBF from 3.39 to 3.92 on the 6-point scale. Federal respondents 
continued to rate both products lower than other jurisdictions with emergency 
management rating both products lower than other disciplines. 
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DHS/FBI Value 


The individual responses to the three questions were combined to form a 
DHS/FBI Value factor; the mean was calculated for both standard and CBF 
responses. 

Mean 


Discipline 

Jurisdiction 

Standard 
Average Value 

CBF Average 
Value 

Other 

Other 

3.2500 

3.9500 

Federal 

3.0125 

3.6190 

State 

3.4167 

4.0625 

County 

3.5000 

4.7500 

Local 

3.3571 

3.9286 

Private Sector 

3.7500 

4.2500 

Total 

3.2448 

3.8673 

Public Health 

Other 

2.8750 

3.5625 

Federal 

3.0000 

3.2500 

State 

2.8333 

3.3214 

County 

3.4833 

3.8333 

Local 

3.6250 

4.0417 

Total 

3.2125 

3.6842 

Law Enforcement 

Other 

3.5000 

4.3750 

Federal 

3.2396 

3.8229 

State 

3.3333 

4.0167 

County 

3.2917 

4.0833 

Local 

3.3000 

3.9150 

Total 

3.2980 

3.9369 

Fire 

Other 

2.7500 

2.7500 

State 

3.2500 

3.0000 

County 

3.2500 

4.0714 

Local 

3.6188 

3.9408 

Total 

3.5250 

3.8906 

EMS 

County 

3.5000 

3.2500 

Local 

3.2083 

4.0000 

Total 

3.2500 

3.8929 

Emergency Management 

Other 

3.6250 

4.0000 

Federal 

2.9063 

3.2188 

State 

3.4375 

4.0000 

County 

3.0833 

3.4167 

Local 

3.6071 

4.3214 


Total 

3.2847 

3.7431 

Total 

Other 

3.1848 

3.7935 

Federal 

3.0972 

3.6364 

State 

3.2722 

3.8895 

County 

3.3205 

3.8269 

Local 

3.4440 

3.9605 

Private Sector 

3.7500 

4.2500 


Total 

3.3143 

3.8556 

Table 11. DHS/FBI 

“Value” factor response demographics. 
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Appendix D reflects the intention of DHS/FBI to define and obtain feedback on 
intelligence product value by requesting responses to the above three questions under the 
heading of “value.” When the three questions are averaged into a “value” factor, county 
EMS and state fire responses reflect a belief that the standard product was of more value, 
“other” fire showed no preference and the twenty-four other combinations of jurisdictions 
and disciplines indicated that the CBF product was of greater value; public health and 
federal responses rated all products lower than other disciplines and jurisdictions. Mean 
results improved 16.33% with the application of CBF from 3.3143 to 3.8556 on the 6- 
point scale. Both “value” and “quality” factors were defined by the DHS/FBI questions 
found in Figure 1 under those headings. 

Regardless of discipline or jurisdiction, the mean of combined perceived value 
and quality factors improved with the application of CBF. 
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Mean 


Discipline 

Jurisdiction 

Standard 
Average Value 

CBF Average 
Value 

Standard 
Average Quality 

CBF Average 
Quality 

Other 

Other 

3.25 

3.95 

3.0833 

3.9167 

Federal 

3.0125 

3.619 

3.2833 

3.75 

State 

3.4167 

4.0625 

3.9091 

4.4444 

County 

3.5 

4.75 

2.6667 

4.6667 

Local 

3.3571 

3.9286 

3.2857 

4.0952 

Private 

Sector 

3.75 

4.25 

3.4444 

4.1111 

Total 

3.2448 

3.8673 

3.413 

4.0355 

Public Health 

Other 

2.875 

3.5625 

2.6296 

3.1852 

Federal 

3 

3.25 

2.3333 

2.6667 

State 

2.8333 

3.3214 

3.4 

3.8889 

County 

3.4833 

3.8333 

3.4667 

3.7556 

Local 

3.625 

4.0417 

3.5 

3.9444 

Total 

3.2125 

3.6842 

3.2222 

3.6341 

Law Enforcement 

Other 

3.5 

4.375 

3.6667 

4.25 

Federal 

3.2396 

3.8229 

3.4267 

3.913 

State 

3.3333 

4.0167 

3.8 

4.2444 

County 

3.2917 

4.0833 

3.6667 

4.1667 

Local 

3.3 

3.915 

3.5867 

4.16 

Total 

3.298 

3.9369 

3.5867 

4.119 

Fire 

Other 

2.75 

2.75 

3.1667 

3.5 

State 

3.25 

3 

2.3333 

2.3333 

County 

3.25 

4.0714 

2.9524 

4 

Local 

3.6188 

3.9408 

3.813 

4.1707 

Total 

3.525 

3.8906 

3.6405 

4.085 

EMS 

County 

3.5 

3.25 

4.6667 

4.6667 

Local 

3.2083 

4 

3.3889 

4 

Total 

3.25 

3.8929 

3.5714 

4.0952 

Emergency 

Management 

Other 

3.625 

4 

3 

4 

Federal 

2.9063 

3.2188 

3.375 

3.625 

State 

3.4375 

4 

3.1667 

3.9167 

County 

3.0833 

3.4167 

3.1481 

3.4074 

Local 

3.6071 

4.3214 

3.7619 

4.5238 


Total 

3.2847 

3.7431 

3.3056 

3.8519 

Total 

Other 

3.1848 

3.7935 

3 

3.6667 

Federal 

3.0972 

3.6364 

3.3273 

3.761 

State 

3.2722 

3.8895 

3.5926 

4.1259 

County 

3.3205 

3.8269 

3.3419 

3.8291 

Local 

3.444 

3.9605 

3.6439 

4.1624 

Private 

Sector 

3.75 

4.25 

3.4444 

4.1111 


Total 

3.3143 

3.8556 

3.4775 

3.9929 


Table 12. Combined DHS/FBI “Quality” and “Value” factor demographics. 


62 
















































5. Analysis DHS/FBI Value/Quality/Standard/CBF Comparisons 

In choosing statistical tests it is important to detennine whether the results are 
dependent or independent. If the means (averages) are based on the same people 
answering all questions, the means are considered dependent, as is the case in this 
research. Another critical step is to determine whether the data obtained is normally 
distributed (parametric), or non parametric. By obtaining a basic knowledge of the 
underlying distribution of a variable, we can make predictions about the results of 
repeated samples. Normal distribution is reflected in a “bell curve.” A review of the data 
in this research shows that it is non parametric. It is similarly important to determine if 
the data is nominal, ordinal, or scaled. Nominal data are labels such as “married” or 
“single” that can be counted, but not be ordered or measured. In this research, 
jurisdiction and discipline are nominal data. Ordinal data are values or observations that 
can be ranked, or have a rating scale attached. You can count and order, but not measure 
ordinal data. Scaled data is a measurement where the distance between any two units of 
measurements is the same, but the zero point is arbitrary, not attached to the rating scale. 
The Likert scales used in this research are ordinal, with positive “whole number” integers 
attached to a rating scale. For dependent, ordinal results as in this research, the Wilcoxon 
test for paired data is appropriate to test the null hypothesis that there is no difference in 
responses to standard and CBF products, in order to detennine if it is appropriate to reject 
the null hypothesis (H„) in favor of the alternative hypothesis (H,). The Wilcoxon test 
computes the difference in scores, ranks those differences in scores, and then computes 
the mean rank for all cases with a negative difference and positive difference. The 
number of scores in each question varies due to the N/A option provided in the DHS/FBI 
surveys that were dropped in this computation. The Wilcoxon signed rank test is similar 
to the t test of central tendency; however it is more robust as it does not assume a normal 
distribution. Given the non parametric distribution of data observed in this data, the 
Wilcoxon signed rank test is a more powerful test than the t test. 
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N 

Mean Rank 

Sum of Ranks 

CBF Timely and Relevant - 
Standard Timely and 

Negative Ranks 

22(a) 

47.32 

1041.00 

Relevant 

Positive Ranks 

76(b) 

50.13 

3810.00 


Ties 

177(c) 




Total 

275 



CBF Clear and Logical - 
Standard Clear and Logical 

Negative Ranks 

23(d) 

48.50 

1115.50 


Positive Ranks 

109(e) 

70.30 

7662.50 


Ties 

148(f) 




Total 

280 



CBF Reliable - Standard 
Reliable 

Negative Ranks 

9(9) 

48.50 

436.50 


Positive Ranks 

147(h) 

80.34 

11809.50 


Ties 

118(i) 




Total 

274 



CBF Satisfying Gaps - 
Standard Satisfying Gaps 

Negative Ranks 

220) 

60.32 

1327.00 


Positive Ranks 

137(k) 

83.16 

11393.00 


Ties 

117(1) 




Total 

276 



CBF Change - Standard 
Change 

Negative Ranks 

15(m) 

56.30 

844.50 


Positive Ranks 

122(n) 

70.56 

8608.50 


Ties 

133(o) 




Total 

270 



CBF Informed - Standard 
Informed 

Negative Ranks 

18(p) 

53.08 

955.50 


Positive Ranks 

120(g) 

71.96 

8635.50 


Ties 

138(r) 




Total 

276 



CBF New Information - 
Standard New Information 

Negative Ranks 

21 (s) 

71.43 

1500.00 


Positive Ranks 

132(t) 

77.89 

10281.00 


Ties 

122(u) 




Total 

275 




a CBF Timely and Relevant < Standard Timely and Relevant 
b CBF Timely and Relevant > Standard Timely and Relevant 
c CBF Timely and Relevant = Standard Timely and Relevant 
d CBF Clear and Logical < Standard Clear and Logical 
e CBF Clear and Logical > Standard Clear and Logical 
f CBF Clear and Logical = Standard Clear and Logical 
g CBF Reliable < Standard Reliable 
h CBF Reliable > Standard Reliable 
i CBF Reliable = Standard Reliable 
j CBF Satisfying Gaps < Standard Satisfying Gaps 
k CBF Satisfying Gaps > Standard Satisfying Gaps 
I CBF Satisfying Gaps = Standard Satisfying Gaps 


m CBF Change < Standard Change 
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n CBF Change > Standard Change 
o CBF Change = Standard Change 
p CBF Informed < Standard Informed 
q CBF Informed > Standard Informed 
r CBF Informed = Standard Informed 
s CBF New Information < Standard New Information 
t CBF New Information > Standard New Information 
u CBF New Information = Standard New Information 

Table 13. DHS/FBI Value/Quality/Standard/CBF comparison ranks. 



CBF 

Timely 

and 

Relevant - 
Standard 
Timely 
and 

Relevant 

CBF Clear 
and 

Logical - 
Standard 
Clear and 
Logical 

CBF 

Reliable - 
Standard 
Reliable 

CBF 

Satisfying 
Gaps - 
Standard 
Satisfying 
Gaps 

CBF 

Change - 
Standard 
Change 

CBF 

Informed - 
Standard 
Informed 

CBF New 
Information 
- Standard 
New 

Information 

z 

-5.356(a) 

-7.826(a) 

-10.280(a) 

-9.017(a) 

-8.825(a) 

-8.562(a) 

-8.542(a) 

Asymp. Sig. (2- 
tailed) 

.000 

.000 

.000 

.000 

.000 

.000 

.000 


a Based on negative ranks, 
b Wilcoxon Signed Ranks Test 


Table 14. DHS/FBI Value/Quality/Standard/CBF Z scores. 



N 

Mean 

Std. Deviation 

Minimum 

Maximum 

Standard Average Quality 

282 

3.4775 

.92315 

1.00 

5.00 

Standard Average Value 

280 

3.3143 

.89596 

1.00 

5.00 

CBF Average Quality 

280 

3.9929 

.83527 

1.00 

5.00 

CBF Average Value 

277 

3.8556 

.89453 

1.00 

5.00 


Table 15. DHS/FBI Value/Quality/Standard/CBF standard deviations. 



N 

Mean Rank 

Sum of Ranks 

CBF Average Quality - Negative Ranks 

25(a) 

65.08 

1627.00 

Standard Average Quality Positive Ranks 

168(b) 

101.75 

17094.00 

Ties 

85(c) 



Total 

278 



CBF Average Value - Negative Ranks 

34(d) 

52.32 

1779.00 

Standard Average Value Positive Ranks 

178(e) 

116.85 

20799.00 

Ties 

64(f) 



Total 

276 




a CBF Average Quality < Standard Average Quality 
b CBF Average Quality > Standard Average Quality 
c CBF Average Quality = Standard Average Quality 
d CBF Average Value < Standard Average Value 
e CBF Average Value > Standard Average Value 
f CBF Average Value = Standard Average Value 

Table 16. DHS/FBI mean Value/Quality/Standard/CBF standard deviations. 
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CBF Average 
Quality - 
Standard 
Average 
Quality 

CBF Average 
Value - 
Standard 
Average Value 

z 

Asymp. Sig. (2-tailed) 

-9.984(a) 

.000 

-10.684(a) 

.000 


a Based on negative ranks, 
b Wilcoxon Signed Ranks Test 

Table 17. DHS/FBI Value/Quality/Standard/CBF Wilcoxon signed ranks test. 


6. Conclusions: Perceived Value and Quality 

The Wilcoxon test conducted examines the hypothesis that there is no difference 
in perceived quality and value between standard and CBF products. This is not the case; 
the observed increased perception in both value and quality would occur fewer than once 
every thousand times if there really was no difference in perceived quality or value 
between the standard and CBF product. The improvement in perceived quality and value 
is statistically significant. While not included in the original hypothesis’, the Wilcoxon 
test on each of the seven questions comprising the DHS/FBI quality and value measures 
would occur fewer than once every thousand times if there really were no difference 
between the standard and CBF products. The ranks chart shows that the CBF product 
was rated higher by an overwhelming margin, from three to sixteen times higher on 
individual variables (questions) and five to six times higher when combined into value 
and quality factors. 

Reject H 0: There is no difference in intelligence product perceived value as 
represented by DHS/FBI customer satisfaction surveys, on average, with the application 
of CBF to open sources in favor of Hi : An intelligence product with CBF to open sources 
is perceived as more valuable based on DHS/FBI customer satisfaction surveys than a 
non- CBF product, on average. 

Reject Ho; There is no difference in intelligence product perceived quality as 
represented by DHS/FBI customer satisfaction surveys, on average, with the application 
of CBF to open sources in favor of Hi : An intelligence product with CBF to open sources 
is perceived as higher quality based on DHS/FBI customer satisfaction surveys than a 
non-CBF product, on average. 
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7. Responses: Technology Acceptance Model 
a. Perceived Usefulness 

The six questions comprising the Technology Acceptance Model 
“Perceived Usefulness” factor of the survey, 

1. Using the intelligence product with hyperlinks in my job would enable me to 
accomplish tasks more quickly. 

2. Using intelligence products with hyperlinks would improve my job performance. 

3. Using intelligence products with hyperlinks would increase my productivity. 

4. Using intelligence products with hyperlinks would enhance my effectiveness on 

the job. 

5. Using intelligence products with hyperlinks would make it easier to do my job 

6. I would find intelligence products with hyperlinks useful in my job. 


were evaluated to detennine the perceived usefulness of hyperlink technology to achieve 
CBF in unclassified intelligence. On the 6-point Likert scale with 6 representing 
“strongly agree,” and 1 “strongly disagree” with the usefulness of hyperlink technology, 
the mean varied from 4.42 to 4.74, with a mode of 5, indicating the respondents belief 


that hyperlink technology was useful in this application. 



Quickly 

Performance 

Productivity 

Effectiveness 

Easier 

Useful 

N Valid 

285 

285 

285 

285 

285 

285 

Missing 

0 

0 

0 

0 

0 

0 

Mean 

4.61 

4.47 

4.42 

4.49 

4.60 

4.74 

Median 

5.00 

5.00 

5.00 

5.00 

5.00 

5.00 

Mode 

5 

5 

5 

5 

5 

5 

Std. Deviation 

1.084 

1.105 

1.162 

1.143 

1.098 

1.082 

Variance 

1.176 

1.222 

1.350 

1.307 

1.206 

1.172 

Skewness 

-.853 

-.617 

-.565 

-.633 

-.754 

-.961 

Std. Error of Skewness 

.144 

.144 

.144 

.144 

.144 

.144 

Kurtosis 

.835 

.336 

-.077 

.181 

.409 

1.030 

Std. Error of Kurtosis 

.288 

.288 

.288 

.288 

.288 

.288 


Table 18. TAM perceived usefulness variable descriptive statistics. 
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When the six components of perceived usefulness are combined, a mean of 4.5538 with a 
mode of 5 was found. 


Perceived Usefulness 


N Valid 

285 

Missing 

0 

Mean 

4.5538 

Median 

4.8333 

Mode 

5.00 

Std. Deviation 

1.02732 

Variance 

1.055 


Table 19. TAM perceived usefulness factor descriptive statistics. 


Variation between disciplines was found, with public health respondents 
finding the hyperlinks useful, but less so than law enforcement and EMS respondents 
whose usefulness mode was 5 on the 6-point Likert scale. Variance is more fully 
explored in correlation, regression and multiple regression segments of this research. 


Discipline 

Average 

Usefulness 

Other 

4.5272 

Public Health 

4.0873 

Law Enforcement 

4.6967 

Fire 

4.5915 

EMS 

4.7381 

Emergency Management 

4.6481 

Total 

4.5538 


Table 20. TAM perceived usefulness discipline responses. 
b. Perceived Ease of Use 

The six questions comprising the Technology Acceptance Model 
“Perceived Ease of Use” factor of the survey, 

1. Learning to utilize intelligence products with hyperlinks would be easy for 
me. 

2. I would find it easy to use intelligence hyperlinks to obtain decision 
making information. 
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3. My interaction with the intelligence product with hyperlinks was clear and 
understandable. 

4. I found the intelligence hyperlinks to be flexible to interact with. 

5. It would be easy for me to become skillful at using intelligence products 
with hyperlinks. 

6. I found the intelligence product with hyperlinks easy to use. 


were evaluated to determine the perceived ease of use of hyperlinks to achieve CBF in 
unclassified intelligence. On the 6-point Likert scale with 6 representing “strongly 
agree,” and 1 “strongly disagree” with the ease of use of hyperlink technology, The mean 
varied from 4.66 to 5.01, with a mode of 5, indicating the respondents belief that 
hyperlink technology was easy to use in this application. 



Easy 

Decision Making 

Clear 

Flexible 

Skillful 

Easy to Use 

N Valid 

285 

285 

285 

285 

285 

285 

Missing 

0 

0 

0 

0 

0 

0 

Mean 

5.01 

4.66 

4.84 

4.68 

4.89 

4.98 

Median 

5.00 

5.00 

5.00 

5.00 

5.00 

5.00 

Mode 

5 

5 

5 

5 

5 

5 

Std. Deviation 

1.026 

1.135 

1.054 

1.067 

1.060 

1.017 

Variance 

1.053 

1.289 

1.110 

1.139 

1.123 

1.035 

Skewness 

-1.183 

-1.004 

-.921 

-.724 

-1.022 

-1.069 

Std. Error of Skewness 

.144 

.144 

.144 

.144 

.144 

.144 

Kurtosis 

.609 

.860 

.713 

.220 

1.078 

1.289 

Std. Error of Kurtosis 

.288 

.288 

.288 

.288 

.288 

.288 


Table 21. TAM perceived ease of use variable descriptive statistics. 


When the six components of perceived ease of use are combined, a mean 
of 4.8439 with a mode of 5 was found. 
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Perceived Ease of Use 


N Valid 

285 

Missing 

0 

Mean 

4.8439 

Median 

5.0000 

Mode 

5.00 

Std. Deviation 

.96145 

Variance 

.924 


Table 22. TAM perceived ease of use factor descriptive statistics. 

Similar to usefulness, variation between disciplines was found in ease of 
use by discipline; public health respondents again found the hyperlinks useful, but less so 
than law enforcement whose mean ease of use response exceeded 5 on the 6-point Likert 
scale with a mode of 5. 


Discipline 

Average Ease 
of Use 

Other 

4.8605 

Public Health 

4.4762 

Law Enforcement 

5.0417 

Fire 

4.7778 

EMS 

4.8333 

Emergency Management 

4.7963 

Total 

4.8439 


Table 23. TAM perceived ease of use discipline responses. 


8. Conclusions: Technology Acceptance 

Survey data that shows an improvement in both usefulness and ease of use 
indicate a broad acceptance of hyperlink technology to achieve CBF across disciplines 
and jurisdictions, with strong evidence of support in all twelve validated technology 
acceptance model questions. With a mode of 5 out of a possible 6 on every question, 
respondents showed overwhelming acceptance of hyperlink technology use in 
unclassified intelligence to achieve CBF. It is important to note that the core of 
technology acceptance in this research is the perception that hyperlinking of contextual 
background to unclassified intelligence products is both useful and easy to use for 
homeland security professionals. 
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Discipline 

Average Ease 
of Use 

Average 

Usefulness 

Other 

4.8605 

4.5272 

Public Health 

4.4762 

4.0873 

Law Enforcement 

5.0417 

4.6967 

Fire 

4.7778 

4.5915 

EMS 

4.8333 

4.7381 

Emergency Management 

4.7963 

4.6481 

Total 

4.8439 

4.5538 

Table 24. TAM 

[ discipline responses. 


Jurisdiction 

Average Ease 
of Use 

Average 

Usefulness 

Other 

4.6319 

4.0972 

Federal 

4.8869 

4.5863 

State 

5.0217 

4.7391 

County 

4.4316 

4.2821 

Local 

4.9373 

4.6410 

Private Sector 

4.7222 

4.8889 

Total 

4.8439 

4.5538 


Table 25. TAM jurisdiction responses. 


All variables demonstrated strong acceptance, even the lowest ranked variable, 
productivity, received a score of 4.42 out of 6. Utilizing TAM to predict the actual use of 
technology, the model below diagrams the strong individual variable inputs to 
“usefulness” and “ease of use” factors that contribute toward the intent to use and 
ultimate predicted usage of CBF in unclassified intelligence. Previous research has 
documented that usefulness is the more important predictor of technology use as no 
matter how easy a technology is to use, it must be useful if it is to be accepted. 
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9. Standard vs. CBF Forced Choice Results 

The final Likert question, “Which type of intelligence product would be of greater 
value to you and your organization?” represented a direct choice for subjects between 
CBF and standard unclassified intelligence products. On the Likert scale forced choice 
model, 1 represented “Non-Hyperlinked,” and 6 “Hyperlinked.” 



Frequency 

Valid Percent 

Valid 

Non-Hyperlinked 

1 

.4 

2 

4 

1.4 

3 

11 

3.9 

4 

20 

7.0 

5 

61 

21.5 

Hyperlinked 

187 

65.8 

Total 

284 

100.0 

Missing 

System 

1 


Total 

285 



Table 26. Standard vs. CBF forced choice frequency. 


N 

Valid 

284 

Missing 

1 

Mean 

5.45 

Median 

6.00 

Mode 

6.00 

Std. Deviation 

.929 


Table 27. Standard vs. CBF forced choice descriptive statistics. 


Valid responses from 284 participants were obtained for this question with 187 or 
65.8% of respondents indicating the strongest possible preference for the hyperlinked 
option offered, producing both a median and mode of 6 with a mean of 5.45; this 
indicates that given a choice, homeland security professionals overwhelmingly preferred 
the CBF product. One response was missing as the survey did not require a response to 
this question. 

Although variation between disciplines is to be expected, all disciplines and 
jurisdictions uniformly preferred the CBF product with means ranging from 5.1 to 5.67 
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on the 6-point Likert scale with law enforcement, fire, local and private sector 
respondents showing the strongest preference for the CBF product. 


Table 28. 


Discipline 

Greater Value 

Other 

5.43 

Public Health 

5.10 

Law Enforcement 

5.60 

Fire 

5.59 

EMS 

5.43 

Emergency Management 

5.33 

Total 

5.45 


Standard vs. CBF forced choice discip 


ine means. 


Jurisdiction 

Greater Value 

Other 

5.38 

Federal 

5.49 

State 

5.41 

County 

5.13 

Local 

5.57 

Private Sector 

5.67 

Total 

5.45 


Table 29. Standard vs. CBF forced choice jurisdiction means. 


10. Survey Comments 

Survey participants were offered an opportunity to make any comments as a last 
question in the survey; 109 participants took the opportunity to make comments. The 
comments reflected the Likert survey results, overwhelmingly positive, with some 
concerns. Consistent with this research, many of the concerns highlighted the need to 
insure that all open source contextual background used be accurate and reliable. Table 3 
includes all free form comments. 


11. Correlation Analysis 

The Technology Acceptance Model (TAM) data in this research is ordinal (rank 
order) and can be ranked into two ordered series. The Spearman R test of correlation is 
an appropriate test that can be computed from ra nk s. In reviewing correlations, all of 

which were significant at the .01 level, it is clear that the forced choice question, “Which 
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type of intelligence product would be of greater value to you and your organization?” has 
greater correlations to all the technology acceptance model questions than DHS/FBI 
quality and value questions. The correlation coefficient can range in value from -1.00 to 
+ 1.00, with +1.00 indicating a perfect, positive linear relationship and -1.00 a perfect, 
inverse relationship. Correlations between two variables does not imply causality, it is 
possible that other variables are causing the correlation in whole or in part. The 
Technology Acceptance Model questions have correlation coefficients ranging from 
+.428 to +.581 with the forced choice “greater value” question, while DHS/FBI value and 
quality correlation coefficients ranged from +.199 to +.339. As the Department of 
Homeland Security and FBI are interested in improving the perceived value of 
unclassified intelligence product for individuals and organizations, these results warrant 
further examination. If the forced choice “greater value” question results are accurate, 
there is more correlation between “greater value” and the validated TAM questions than 
the DHS/FBI quality and value questions currently in use. 

Technology Acceptance Model questions demonstrated strong intra factor 
correlations between “Usefulness” questions with r values ranging from .724-.880, 
consistent with prior validating research on this model and contributing to a conclusion 
that the minor adaptations of original TAM questions necessary for this research 
maintained that validity. TAM “Ease of Use” showed similarly strong intra factor 
correlations ranging between .705-.878 supporting prior research. Discriminate validity 
is evidenced by the low number of item correlations (2/30) that were higher between 
“Usefulness” and “Ease of Use” than within “Usefulness” and “Ease of Use.” These 
observations lead to the conclusion that the questions represent two separate factors and 
that the questions discriminate between these two factors. 

This conclusion is supported through a single summary statistic, Cronbach’s 
Alpha analysis of reliability: 


Cronbach's 

Alpha 

N of Items 

.965 

6 


Table 30. Cronbach’s Alpha “Usefulness” reliability statistics. 
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A Cronbach’s Alpha analysis of reliability can be utilized to determine to what 
degree a survey maker was successful in constructing questions that measure a person’s 
opinion. It provides a lower bound for the true reliability of a survey, analyzing whether 
the survey will differ because respondents have different opinions, as opposed to 
confusion or multiple interpretations of survey questions. The statistic is determined 
based on the number of items in a survey, and measurements of covariance ratios. 


Cronbach's 

Alpha 

Cronbach's 
Alpha Based 
on 

Standardized 

Items 

N of Items 

.957 

.958 

6 


Table 31. Cronbach’s Alpha “Ease of Use” reliability statistics. 


Cronbach's 

Alpha 

Cronbach's 
Alpha Based 
on 

Standardized 

Items 

N of Items 

.818 

.824 

3 


Table 32. Cronbach’s Alpha “Quality” reliability statistics. 


Cronbach's 

Alpha 

Cronbach's 
Alpha Based 
on 

Standardized 

Items 

N of Items 

.925 

.925 

4 


Table 33. Cronbach’s Alpha “Value” reliability statistics. 


Strong intra factor reliability is indicated in the Cronbach’s Alpha of each of the 
four factors utilized in this survey with ranges from .818 to .965. 

The three questions representing the DHS/FBI “Quality” factor showed much 
lower intra factor correlations than TAM factors, ranging between .489 and .679. The 
“Quality” factor questions were almost equally correlated with DHS/FBI “Value” 
questions with correlations ranging between .448-.645, a possible indication that the 
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DHS/FBI quality and value factor questions are not measuring two separate factors of 
value and quality; this warrants further research. 

The four questions within the DHS/FBI “Value” factor had correlations between 
.685-.767, showing stronger correlations than the DHS/FBI “Quality” measurements, and 
indicating that the “value” questions were more closely measuring the same factor. 
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All Correlations are significant at the 0.01 level (1-tailed). 

Table 34. Pearson one tailed correlations. 


TAM Usefulness Factor 

vl Quickly 

v2 Performance 

v3 Productivity 

v4 Effectiveness 

v5 Easier 

v6 Useful 
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TAM Ease of Use Factor 

v7 Easy 

v8 Decision Making 

- Cleaf 
vlO Flexible 
jSkillfli 

vl2 Easy to Use 

v 13 Greater V alue 

DHS/FBI Quality Factor 

vl4 Hyperlinked Timely and Relevant 
vl5 Hyperlinked Clear and Logical 
vl6 Hyperlinked Reliable 

DHS/FBI Value Factor 

vl7 Hyperlinked Satisfying Gaps 
v 18 Hyperlinked Change 

v 19 Hyperlinked Informed 

v20 Hyperlinked New Information 
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12. Regression Analysis 

Regression analysis investigates the relationships between variables. It can be 
used to seek the causal effect of one variable upon another and estimates the statistical 
significance of or degree of confidence in, relationships. In looking at the impact of one 
variable upon another, we are conducting “simple regression analysis.” “Multiple 
regression” estimates the effect of each on a dependent variable and helps explain the 
impact of multiple simultaneous variables on another single variable. R" is the statistic 
that measures the amount of total variation that is explained by the variable. 

In this research, multiple regression analysis with the dependent variable of 
“Which type of intelligence product would be of greater value to you and your 
organization?” was conducted with technology acceptance factors of “Usefulness” and 
“Ease of Use,” along with the DHS/FBI customer satisfaction factors of “quality” and 
“value” in order to estimate the causal effect of the different variables within each of 
these factors on increasing the value of intelligence products. 

In determining the type of multiple regression analysis that is appropriate, it is 
important to screen the data to determine if it is linear in nature. The TAM “Usefulness” 
graph with independent variables and “greater value” dependent value demonstrates that 
this data is linear, indicating that a linear multiple regression is appropriate. The 
observed and expected cumulative probabilities can be observed to follow the straight 
line representing exact matches of observed and expected values. 
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13. Multiple Regression Analysis Technology Acceptance Usefulness 
Factor 


Normal P-P Plot of Regression Standardized Residual 


Dependent Variable: greater value 



0.0 0.2 0.4 0.6 0.8 1.0 


Observed Cum Prob 

Greater value variable normal P-Plot of regression standardized residual. 


Model 

R 

R Square 

Adjusted R 
Square 

Std. Error of 
the Estimate 

1 

.583(a) 

.339 

.325 

.762 


a Predictors: (Constant), Useful, Quickly, Productivity, Easier, performance, 
Effectiveness 

b Dependent Variable: greater value 

Table 35. Multiple Regression “Usefulness” model summary. 


Model 


Sum of 
Squares 

df 

Mean Square 

F 

Sig. 

1 

Regression 

83.074 

6 

13.846 

23.815 

.000(a) 

Residual 

161.628 

278 

.581 



Total 

244.702 

284 





a Predictors: (Constant), Useful, Quickly, Productivity, Easier, performance, Effectiveness 
b Dependent Variable: greater value 


Table 36. Multiple Regression “Usefulness” anova. 
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Model 


Unstandardized 

Coefficients 

Standardized 

Coefficients 

t 

Sig. 

B 

Std. Error 

Beta 

1 

(Constant) 

3.180 

.215 


14.817 

.000 

Quickly 

-.106 

.083 

-.124 

-1.275 

.204 

Performance. 

.030 

.102 

.035 

.290 

.772 

Productivity 

-.096 

.100 

-.120 

-.957 

.339 

Effectiveness 

.254 

.105 

.313 

2.420 

.016 

Easier 

.084 

.098 

.099 

.855 

.393 

Useful 

.323 

.089 

.377 

3.634 

.000 


a Dependent Variable: greater value 

Table 37. Multiple Regression “Usefulness” coefficients. 


R Square is an estimate of how well the model evaluated explains the population 
variance of the dependent variable, in this case “Which type of intelligence product 
would be of greater value to you and your organization?” In this analysis, all the 
questions in the TAM “Usefulness” factor together explain only 1/3 (.339) of the 
dependent variable, not a substantial predictor of “greater value.” 

It is useful to compare the “Regression” sum of squares with the “Residual” sum 
of squares. If the Regression is large in relation to Residual, the model being evaluated 
can be said to account for a large portion of the dependent variable variance. That is not 
the case in “Usefulness,” the relatively high residual indicates that this model fails to 
explain much of the variation, and calls for examination of other variable sets. 

Another critical value in this multiple regression is “t;” the t statistic indicates the 
relative importance of each variable (question) in the multiple regression model tested. 
Values substantially below -2.00 or above 2.00 may be considered useful predictors. In 
this analysis, “Using intelligence products with hyperlinks would enhance my 
effectiveness,” and “I would find intelligence products with hyperlinks useful in my job” 
are useful predictors, as both have t above 2 and are significant at the .05 and .01 level 
respectively. The other variables (questions) in the “Usefulness” factor did not 
substantially contribute. 

A multivariate analysis utilizing only the two variables shown in the full 

“Usefulness” factor analysis to have a t statistic over 2 demonstrates that these two 

variables account for almost all the variance predicted by the six variables in 
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“Usefulness.” The R Square in this case actually increases from .339 to .377, indicating 
the two questions alone better explain the dependent variable variance than all six 
questions together, although the two question model still fails to explain much of the 
variation. 


Model 

R 

R Square 

Adjusted R 
Square 

Std. Error of 
the Estimate 

1 

.614(a) 

.377 

.373 

.735 


a Predictors: (Constant), easy to use, decision making 
b Dependent Variable: greater value 


Table 38. “Usefulness” two variable multiple regression model summary. 


Model 


Sum of 
Squares 

df 

Mean Square 

F 

Sig. 

1 

Regression 

92.354 

2 

46.177 

85.475 

.000(a) 

Residual 

152.348 

282 

.540 



Total 

244.702 

284 





a Predictors: (Constant), easy to use, decision making 
b Dependent Variable: greater value 


Table 39. “Usefulness” two variable multiple anova. 


14. Multiple Regression Analysis Technology Acceptance Ease of Use 
Factor 


Model 

R 

R Square 

Adjusted R 
Square 

Std. Error of 
the Estimate 

1 

.623(a) 

.388 

.375 

.734 


a Predictors: (Constant), easy to use, decision making, Flexible, Easy, Skillful, and 
Clear 

b Dependent Variable: greater value 

Table 40. Multiple Regression “Ease of Use” model summary. 


Model 


Sum of 
Squares 

df 

Mean Square 

F 

Sig. 

1 

Regression 

95.036 

6 

15.839 

29.421 

.000(a) 

Residual 

149.666 

278 

.538 



Total 

244.702 

284 





a Predictors: (Constant), easy to use, decision making, Flexible, Easy, Skillful, and Clear 
b Dependent Variable: greater value 


Table 41. Multiple Regression “Ease of Use” anova. 
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Model 


Unstandardized 

Coefficients 

Standardized 

Coefficients 

t 

Sig. 

B 

Std. Error 

Beta 

1 

(Constant) 

2.503 

.228 


10.975 

.000 

Easy 

.146 

.093 

.161 

1.569 

.118 

decision making 

.125 

.060 

.153 

2.097 

.037 

Clear 

-.033 

.104 

-.038 

-.323 

.747 

Flexible 

.095 

.080 

.109 

1.183 

.238 

Skillful 

.026 

.100 

.029 

.254 

.800 

easy to use 

.248 

.110 

.271 

2.261 

.025 


a Dependent Variable: greater value 

Table 42. Multiple Regression “Ease of Use” coefficients. 


Although slightly higher than “Usefulness,” “Ease of Use” R Square of .388 does 
not substantially predict the variance of “greater value.” The large Regression compared 
to Residual will cause us to continue to look for variance sets that better explain “greater 
value” variance. Similar to “Usefulness,” only two variables, “I would find it easy to use 
intelligence hyperlinks to obtain decision making information” and “I found the 
intelligence product with hyperlinks easy to use” have t statistics above 2, making them 
useful predictors of dependent variable variance. 


15. Multiple Regression Analysis DHS/FBI Quality Factor 


Model 

R 

R Square 

Adjusted R 
Square 

Std. Error of 
the Estimate 

1 

.375(a) 

.141 

.131 

.876 


a Predictors: (Constant), H reliable, H timely and relevant, H clear and logical 
b Dependent Variable: greater value 

Table 43. Multiple Regression DHS/FBI quality factor model summary. 


Model 


Sum of 
Squares 

df 

Mean Square 

F 


1 

Regression 

33.687 

3 

11.229 

14.624 

.000(a) 

Residual 

205.783 

268 

.768 



Total 

239.471 

271 





a Predictors: (Constant), H reliable, H timely and relevant, H clear and logical 
b Dependent Variable: greater value 


Table 44. Multiple Regression DHS/FBI quality factor anova. 
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Model 


Unstandardized 

Coefficients 

Standardized 

Coefficients 

t 

Sig. 

B 

Std. Error 

Beta 

1 

(Constant) 

3.763 

.269 


14.000 

.000 

H timely and relevant 

-.040 

.064 

-.046 

-.625 

.532 

H clear and logical 

.115 

.089 

.114 

1.286 

.199 

H reliable 

.327 

.083 

.312 

3.951 

.000 


a Dependent Variable: greater value 

Table 45. Multiple Regression DHS/FBI quality variable coefficients. 


As might be expected after reviewing the bivariate correlation results that showed 
the individual correlations of the technology acceptance variables to have stronger 
correlations with “greater value” than the DHS/FBI Quality and Value factors, the 
multiple regression analysis of “DHS/FBI Quality” has less explanatory power than the 
TAM factors. The “Quality” R Square indicates that only 14% of dependent variable 
variance is explained by this factor, the high Regression to Residual ratio causes us to 
look for other explanations, and only one of the three questions, “The product is reliable; 
i.e., sources well documented and reputable” has a t statistic above 2, albeit the highest 
yet at 3.951. A review of the “reliable” variable alone shows only a minor drop in 
variance explanation from 14% for all three variables to 13% with the single variable. 


Model 

R 

R Square 

Adjusted R 
Square 

Std. Error of 
the Estimate 

1 

.363(a) 

.132 

.128 

.874 


a Predictors: (Constant), H reliable 
b Dependent Variable: greater value 


Table 46. Multiple Regression DHS/FBI quality “reliable” variable model summary. 

16. Multiple Regression Analysis DHS/FBI Value Factor 


Model 

R 

R Square 

Adjusted R 
Square 

Std. Error of 
the Estimate 

1 

.325(a) 

.106 

.092 

.894 


a Predictors: (Constant), H new information, H change, H satisfying gaps, H informed 
b Dependent Variable: greater value 

Table 47. Multiple Regression DHS/FBI value factor model summary. 
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Model 


Unstandardized 

Coefficients 

Standardized 

Coefficients 

t 

Sig. 

B 

Std. Error 

Beta 

1 

(Constant) 

4.156 

.240 


17.314 

.000 

H satisfying gaps 

.061 

.096 

.066 

.640 

.523 

H change 

.004 

.091 

.004 

.040 

.968 

H informed 

.048 

.108 

.051 

.448 

.654 

H new information 

.217 

.098 

.226 

2.221 

.027 


a Dependent Variable: greater value 

Table 48. Multiple Regression DHS/FBI value factor variable coefficients. 



Minimum 

Maximum 

Mean 

Std. 

Deviation 

N 

Predicted Value 

4.49 

5.81 

5.45 

.305 

273 

Residual 

-4.043 

1.514 

.000 

.887 

273 

Std. Predicted Value 

-3.149 

1.181 

.000 

1.000 

273 

Std. Residual 

-4.523 

1.693 

.000 

.993 

273 


a Dependent Variable: greater value 


Table 49. Multiple Regression DHS/FBI value factor residual statistics. 


The “Value” factor showed the lowest explanatory power with an R Square of 
.106, or less than 11% of dependent variable variance. Only one variable, “The product 
would identify new infonnation associated with pending matters or offered insights into 
information that could change the working premise in a program or initiative” had a t 
statistic above 2, and Predicted/Residual indicates a poor explanation of dependent 
variable variance 


17. Multiple Regression Analysis Summary 

The analysis of how well each factor explains the variance of “Which type of 
intelligence product would be of greater value to you and your organization?” shows that 
“Ease of Use,” the most significant factor with an R Square of 0.388, explains less than 
40% of “greater value” variance. The TAM factors of Usefulness and Ease of Use 
demonstrated dramatically greater explanatory power than the DHS/FBI Quality and 
Value factors. 
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Factor 

R Square 



Usefulness 

0.339 

Ease of Use 

0.388 

Quality 

0.141 

Value 

0.106 


Table 50. Multiple Regression Usefulness/Ease of Use/Quality/Value R square. 


The following diagram summarizes both explanatory R Square (R 2 ) and 
individual variable correlations (R) with greater value. In this diagram it is clear that the 
highest correlated DHS/FBI variable, “The product is reliable; i.e., sources well 
documented and reputable” equals the correlation of “usefulness,” the lowest TAM 
variable at .339, supporting findings that overall technology acceptance modeling is a 
better predictor of “greater value” ratings than current DHS/FBI questions. 
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Figure 7. Factor multiple regression and individual variable correlation coefficient 

relationships with “greater value” variable. 


Conducting a multiple regression analysis with the individual t statistics that were 
over 2 for each factor yielded the largest R square at .416, indicating that this set of 
variables accounts for approximately 41% of “greater value” variance. “I found the 
intelligence product with hyperlinks easy to use” has the only substantial t statistic in this 
analysis; at 5.223, it still explains less dependent variable variance than unknown 
variables as indicated by the constant t statistic of 8.331 
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Model 

R 


Adjusted R 
Square 

Std. Error of 
the Estimate 

1 

.645(a) 

.416 

.403 

.725 


a Predictors: (Constant), Hyperlinked New Information, Easy to Use, Hyperlinked 
Reliable, Effectiveness, Decision Making, Useful 
Table 51. t statistic > 2 multiple regression model summary. 


Model 


Sum of 
Squares 

df 

Mean Square 

F 

Sig. 

1 

Regression 

99.629 

6 

16.605 

31.559 

.000(a) 

Residual 

139.954 

266 

.526 



Total 

239.582 

272 





a Predictors: (Constant), Hyperlinked New Information, Easy to Use, Hyperlinked Reliable, 
Effectiveness, Decision Making, Useful 
b Dependent Variable: Greater Value 


Table 52. t statistic > 2 multiple regression anova. 




Unstandardized 

Coefficients 

Standardized 

Coefficients 



Model 


B 

Std. Error 

Beta 

t 

Sig. 

1 

(Constant) 

2.213 

.266 


8.331 

.000 


Useful 

.076 

.084 

.086 

.901 

.369 


Effectiveness 

.108 

.071 

.129 

1.523 

.129 


Decision Making 

.073 

.065 

.086 

1.117 

.265 


Easy to Use 

.369 

.071 

.387 

5.223 

.000 


Hyperlinked Reliable 

.023 

.063 

.022 

.370 

.711 


Hyperlinked New 
Information 

.027 

.058 

.028 

.466 

.642 


a Dependent Variable: Greater Value 

Table 53. t statistic > 2 coefficients. 


Reviewing “Easy to Use” alone shows an R Square of .355, that largest single 
explanatory variable; continued review of Regression/Residual shows a high Residual 
and the t statistic shows that this variable is almost equal to the constant. 


Model 

R 

R Square 

Adjusted R 
Square 

Std. Error of 
the Estimate 

1 

.596(a) 

.355 

.353 

.747 


a Predictors: (Constant), Easy to Use 


Table 54. “Easy to Use” model summary. 
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Model 


Sum of 
Squares 

df 

Mean Square 

F 


1 

Regression 

86.826 

1 

86.826 

155.640 

.000(a) 

Residual 

157.876 

283 

.558 



Total 

244.702 

284 





a Predictors: (Constant), Easy to Use 
b Dependent Variable: Greater Value 


Table 55. “Easy to Use” anova. 


Model 


Unstandardized 

Coefficients 

Standardized 

Coefficients 

T 

Sig. 

B 

Std. Error 

Beta 

1 

(Constant) 

2.750 

.221 


12.420 

.000 

Easy to Use 

.544 

.044 

.596 

12.476 

.000 


a Dependent Variable: Greater Value 

Table 56. “Easy to Else” coefficients. 


The multiple variable regression analysis of the variables in this research does not 
substantially explain responses to “Which type of intelligence product would be of 
greater value to you and your organization?” The multicollinearity of the results 
indicates that there may be variables outside of those examined in this research that better 
explain “greater value” variance. Further research to produce validated DHS/FBI 
customer satisfaction survey questions is warranted. 


E. CONCLUSIONS 

Previous theory of reasoned action and technology acceptance model research has 
provided empirical evidence of the relationship between TAM ease of use and usefulness 
factors and actual system use. The demonstrated validity of Technology Acceptance 
Model questions regarding “Usefulness” and “Ease of Use”, combined with 
overwhelming acceptance of the hyperlink technology in this research strongly predicts 
the acceptance of hyperlink technology to achieve CBF in unclassified intelligence across 
the broad range of homeland security disciplines and jurisdictions, extending the 
application of TAM to homeland security settings. In this research, homeland security 
professionals have demonstrated a clear intention to use hyperlinks to achieve CBF, 
strongly predicting actual use. 
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This predicted acceptance, paired with definitive feedback that the CBF product 
would be of greater value to organizations and the statistically significant improvements 
in DHS/FBI “Value” and “Quality” factors indicates that this concept has the potential to 
significantly improve the provision of unclassified intelligence to millions of homeland 
security professionals. CBF has been demonstrated to be one tool that can assist in 
meeting the demands and recommendations of the Markle Foundation, NTR, the 9/11 
Commission, California Police Chiefs, DHS, Major City Police Chiefs, the CIA, the 
National Strategy for Homeland Security, the National Infrastructure Advisory Council, 
and the RAND Corporation cited in this research. CBF of unclassified intelligence for 
NTR contributes toward: 

• “...news.. .in context.” (9/11 Commission, cited on page 1 of this research) 

• “...adequate context for homeland security providers to effectively utilize 
information...specific, tailored for each local entity, rapidly disseminated, and 
does not overburden recipients with vague or irrelevant information.” (Markle 
Foundation, page 1) 

• Improves law enforcement perception of intelligence quality and value. 
(California Police Chiefs, page 2) 

• Is a “better method of sharing intelligence with state and local law enforcement 
agencies?” (DHS, page 2) 

• Contributes toward providing “adequate...background...for 700,000 officers on 
the street.” (Major City Police Chiefs, page 6) 

• “...harness(es) the power of information technology...” (Markle Foundation, 
page 8) 

• Provides “background and contextual information available at...policymaker’s 
fingertips in a timely fashion...” (CIA, pagel 1) 

• Moves toward “full integration of (the) private sector into the intelligence cycle.” 
(National Infrastructure Advisory Council, page 16) 

• Encourages “effective partnership with state and local government and the private 
sector...” (National Strategy for Homeland Security, page 15) 

• Is “tailored to meet requests from government departments, police, and private 
industry.” (RAND, page 25) 
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Research has found that even well educated decision makers who understand the 
power of statistical testing often rely on analogies or metaphors to change deeply held 
beliefs, analogical reasoning combined with statistical analysis is more effective in 
changing those beliefs than statistics alone. 78 

In describing the power of context in terrorism decision making Professor Fathali 
Moghaddam contends that “..terrorism is explained by the power of context...” and 
provides the following analogy: 

...although John could choose to wear a bright red and yellow tie at his 
uncle’s funeral in New York, he is very likely to choose a dark, somber tie 
instead; just as when John gets married, his future wife could choose to 
follow an alternative fashion trend and wear a black dress at their 
wedding, but she is far more likely to wear a white dress. In each case, the 
individual choices, John deciding to wear a dark tie at a funeral and his 
bride deciding to wear a white dress at their wedding, are made with some 
measure of free will and we can hold the individuals responsible for their 
choices, but we must also accept that their choices were heavily influenced 
by the social contexts (of a funeral and a wedding). 79 

Another contextual metaphor/urban legend about the FBI describes how J. Edgar 
Hoover demanded short memos with wide margins (in order to make notations), then 
received a memo with narrow margins. He promptly returned the memo with the 
notation, “Watch the borders.” Subordinates promptly sent hundreds of agents to guard 
our national borders with Canada and Mexico! 

Whether it is as mundane as dressing appropriately for a funeral, as expensive as 
sending hundreds of agents to the borders, or as critical as meeting the unclassified 
intelligence needs of millions of homeland security professionals, context matters. This 
research demonstrates that through CBF it is practical for the Intelligence Community to 
provide NTR the context required to make good decisions. Context that is consistent 
with the henneneutic of the originating agency, is immediately available to decision 
makers, is “fused” to keywords, is available to over 87,000 jurisdictions, improves both 

quality and value of intelligence products and utilizes technology that is widely accepted. 

78 Nagy, Thomas, Theresa Jefferson and Jamal Altorkistani, Selling Proven Methods for Improving 
the Usability of the Web: Experimental Evidence of the Benefits of Metaphors for Supporting Heuristic 
Evaluation, http://home.gwu.edu/~nagy/Hfweb99.htm. ( accessed October 18, 2006), 1. 

79 Moghaddam, Fathali, From the Terrorists' Point of View What They Experience and Why They 
Come to Destroy (Westport, London: Praeger Security International, 2006), 11 
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IV. RECOMMENDATIONS 


Research has outlined the critical impact of context on decision making, the 
importance of intelligence in homeland security, the need to involve millions of diverse 
homeland security professionals in asymmetric conflict, and the importance of 
technology in intelligence operations. Statistical analysis of the data indicates that 
hyperlinking unclassified intelligence products to open source contextual background 
(CBF) increases the perceived value and quality of that intelligence. Given a choice, 
homeland security professionals overwhelmingly preferred a CBF product, 268 to 16. 
Observed increases in perceived value and quality and clear preference for CBF in 
intelligence products make it clear that CBF positively impacts information sharing as 
demanded by national directives, initiatives and homeland security professionals. A 
critical evaluation of these findings utilizing technology acceptance model research 
indicates that not only does CBF improve unclassified intelligence products; the use of 
hyperlink technology to achieve CBF is widely accepted by NTR. Strong ease of use and 
usefulness findings across disciplines and jurisdictions predict that NTR will use this 
technology if employed in unclassified intelligence production and distribution. In order 
to improve information sharing to and between NTR as demanded, the following 
recommendations should be considered: 

• Unclassified intelligence producers with the capability should immediately 
fuse vetted, accurate, open source contextual background to their intelligence 
products through the use of hyperlink technology. 

Homeland security professionals have expressed acceptance of hyperlink 
technology in unclassified intelligence distribution. Intelligence producers such as 
NYPD have successfully integrated other technologies such as PowerPoint into 
unclassified intelligence production and advanced contextual background technology 
such as “Intelink,” “K2,” “Profiler,” and “Autonomy” are successfully used in classified 
intelligence systems. This Department of Homeland Security sponsored research has 
demonstrated the acceptance of technology across a broad range of homeland security 
disciplines and the effectiveness of TAM in evaluating technology prior to widespread 
homeland security application. 
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• DHS should immediately sponsor research to detennine what other 
technologies, if any, would be acceptable and improve perceived value and 
quality in unclassified intelligence products. More sophisticated contextual 
background systems such as K2, Profiler or Autonomy, community of interest 
search systems prioritizing results based on the search history of homeland 
security professionals in similar disciplines/jurisdictions/geographic areas, 
discipline specific products, products with discipline specific contextual 
background hyperlinks and PowerPoint are all technologies that are utilized 
by some jurisdictions or in applications outside unclassified intelligence that 
should be evaluated utilizing TAM for broad homeland security application. 
Technology that is found useful and easy to use should be considered for 
widespread application to improve information sharing. 


This research has demonstrated that statistically significant improvements in 
DHS/FBI defined perceived value and quality factors along with strong TAM ease of use 
and usefulness factor ratings are possible with the addition of open source contextual 
background to unclassified intelligence. NTR found the simple addition of contextual 
background to unclassified intelligence improved that intelligence, contributing to 
improved information sharing as demanded by national directives, initiatives and 
homeland security professionals. 

• The Department of Homeland Security should sponsor research into what 
other information, data or intelligence components would improve perceived 
quality and value of unclassified intelligence products for NTR. The preferred 
length of unclassified intelligence, the optimal amount of contextual 
background, the inclusion of sources for further information, citations, and the 
fusion of related online training to unclassified intelligence products are 
examples of changes to unclassified intelligence that should be evaluated 
utilizing TAM. 


Interpol acts as a distributor of unclassified intelligence products produced by all 
member agencies to other member nations as opposed to being a producer of unclassified 
intelligence themselves and regional organizations in the U.S. have successfully 
distributed unclassified intelligence produced by different disciplines and jurisdictions 
(such as the California OES distribution of NYPD unclassified intelligence Power 
Points). 
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• The Department of Homeland Security should investigate the feasibility of 
distributing the unclassified intelligence products of allied agencies such as 
NYPD in distribution systems under their control a la Interpol. 


The regional fusion center concept has demonstrated an ability to coordinate 
multiple disciplines in counter terrorism efforts. 

• The Department of Homeland Security should continue to support the fusion 
center concept, encouraging the use of these centers to “tailor” intelligence to 
the unique needs of their multidisciplinary partners and geographic area of 
responsibility, consistent with study recommendations. Additionally, DHS 
should sponsor research into additional methods of geographic “tailoring” of 
unclassified intelligence. 


Analysis of responses to DHS/FBI customer satisfaction survey questions 
produced results that reflect low correlation with “greater value;” there appears to be 
multicollinearity with multiple questions measuring the same phenomena along with 
some individual questions that do not substantially contribute to ‘value” and “quality” 
factors with potential overlap between questions and factors, while previously validated 
TAM questions regarding “usefulness” and “ease of use” continued to demonstrate 
statistical validity. 

• DHS/FBI should continue to collect data on customer satisfaction, partnering 
with the Naval Postgraduate School, Center for Homeland Defense and 
Security, providing ongoing data to the university and working collaboratively 
to analyze data, refine customer satisfaction (intelligence feedback) survey 
questions, and leverage the academic rigor and diverse knowledge, skills and 
abilities of staff and students to improve unclassified intelligence. 


The Naval Postgraduate School, Center for Homeland Defense and Security has 
established itself as a successful educational model, bringing well educated, experienced 
and diverse professionals together in a collaborative, focused effort to improve homeland 
defense and security. Every discipline and jurisdiction represented in the cohorts has a 
demonstrated need for unclassified intelligence, and this research reflects a desire and 
ability to receive improved unclassified products. The need for quality unclassified 

intelligence is a significant commonality across the diverse disciplines, jurisdictions, and 
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positions of CHDS students; the vast majority of employees in every parent agency do 
not have security clearances, potentially making unclassified intelligence needs the most 
significant common denominator at CHDS. It is practical to leverage the knowledge, 
skills and abilities of CHDS students along with the sponsorship of DHS to improve 
unclassified intelligence. 

• The Naval Postgraduate School, Center for Homeland Defense and Security 
should initiate a specific course on unclassified intelligence, utilizing the 
collective knowledge, skills and abilities of cohort members to obtain 
feedback and produce original research on network centric strategies to 
improve unclassified intelligence production and distribution for respective 
employing agencies. 

As the Center for Homeland Defense and Security approaches the graduation of 
200 Homeland Security Professionals, an opportunity to leverage this critical mass of 
field tested, academically qualified and geographically dispersed research partners exists 
for DHS. Each participant of CHDS, individually selected by DHS to represent 
jurisdictional, disciplinary, and geographic diversity within the practice of U.S. homeland 
security has demonstrated a significant commitment toward ongoing improvements in 
homeland security. More than a university based educational program, CHDS’ unique 
access to these diverse professionals, contacts within multiple levels of government and 
developing university partnerships provides an opportunity to take the next step, 
combining these assets to further assist homeland security through increased applied 
academic research. Unclassified intelligence research provides an opportunity to 
significantly impact homeland security through the millions of NTR without security 
clearances who make the bulk of public contacts every day. While this “Wal-Mart” 
homeland security intelligence for everyone, everywhere, may lack the public appeal of 
top secret, need to know intelligence research, it addresses the needs of those who 
actually get the work done on a day to day basis, the patrol officers responsible for 80% 
of all arrests, firefighters, emergency medical technicians, and public health professionals 
responding daily to emergency calls for service, and the two million security 
professionals protecting 85% our nation’s infrastructure. 
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• CHDS should immediately initiate an unclassified intelligence research grant 
in partnership with DHS, utilizing current and former students and faculty to 
integrate the recommendations contained in this research, identifying specific 
actions that support the national vision and demands of NTR. 
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APPENDIX A. CONTEXTUAL BACKGROUND WEBSITES 


http://newark.fbi.gOv/dojpressrel/2006/nk030206usa.htm 
http://www.tkb.org/Category.jsp?catID=6 

http://www.mipt.org/pdf/Military-Guide-to-Terrorism-Twenty-First-Century.pdf 

http://www.nal.usda.gov/awic/legislat/pll02346.htm 

http://www.tkb.org/KeyLeader.jsp?memID=6114 

http ://www. tkb. org / Group .j sp? groupID=4234 

http ://www. fbi. gov/wanted/ alert/sandiego_da.htm 

http ://www. tkb .org/Group .j sp?groupID= 14 

http ://www. tkb .org/Group .j sp?groupID=41 

http://www.fbi.gov/publications/terror/terror99.pdf 
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APPENDIX B. COUNTS AND PERCENTAGES 



% within 
discipline 


% within 
jurisdiction 


% of Total 


Jurisdiction 

Other 

Federal 

State 

County 

Local 

Private 

Sector 

5 

21 

12 

1 

7 

3 

10.2% 

42.9% 

24.5% 

2.0% 

14.3% 

6.1% 

20.8% 

37.5% 

26.1% 

2.6% 

6.0% 

100.0% 

1.8% 

7.4% 

4.2% 

.4% 

2.5% 

1.1% 

9 

2 

10 

15 

6 

0 

21.4% 

4.8% 

23.8% 

35.7% 

14.3% 

.0% 

37.5% 

3.6% 

21.7% 

38.5% 

5.1% 

.0% 

3.2% 

.7% 

3.5% 

5.3% 

2.1% 

.0% 



4.0% 

25.0% 

15.0% 

6.0% 

50.0% 

.0% 

16.7% 

44.6% 

32.6% 

15.4% 

42.7% 

.0% 

1.4% 

8.8% 

5.3% 

2.1% 

17.5% 

.0% 

2 

0 

1 

7 

41 

0 

3.9% 

.0% 

2.0% 

13.7% 

80.4% 

.0% 

8.3% 

.0% 

2.2% 

17.9% 

35.0% 

.0% 

.7% 

.0% 

.4% 

2.5% 

14.4% 

.0% 

0 

0 

0 

1 

6 

0 

.0% 

.0% 

.0% 

14.3% 

85.7% 

.0% 

.0% 

.0% 

.0% 

2.6% 

5.1% 

.0% 

.0% 

.0% 

.0% 

.4% 

2.1% 

.0% 


49 


100 . 0 % 

17.2% 


17.2% 


42 


100 . 0 % 


14.7% 


14.7% 


100 


100 . 0 % 


35.1% 


35.1% 


51 


100 . 0 % 



11.1% 

22.2% 

22.2% 

25.0% 

19.4% 

.0% 

16.7% 

14.3% 

17.4% 

23.1% 

6.0% 

.0% 

1.4% 

2.8% 

2.8% 

3.2% 

2.5% 

.0% 

24 

56 

46 

39 

117 

3 

8.4% 

19.6% 

16.1% 

13.7% 

41.1% 

1.1% 

100.0% 

100.0% 

100.0% 

100.0% 

100.0% 

100.0% 

8.4% 

19.6% 

16.1% 

13.7% 

41.1% 

1.1% 


100 . 0 % 


2.5% 


2.5% 


36 


100 . 0 % 


12 . 6 % 


12 . 6 % 

285 


100 . 0 % 


100 . 0 % 


100 . 0 % 
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APPENDIX C. COMMENTS 


30. Comments? 

# 

Response 

1 

It was not clear early on that we were to compare the difference 
between info with, and without hyperlinks. That should be pointed out 
early. 

2 

1 think you should add a sub-category to distinguish LE intel from other 

LE groups. The info in the bulletin isn't going to change anything most 
intel units are going to do because they should already have info, and it 
won't be considered timely. Now, non-intel might give another 
response. 

3 

The hyper linked document allows me to get more information in the 
areas 1 need. 

4 

Easy to understand and source, especially the hyperlinked version. 
Excellent example of quality intelligence dissemination at an 
unclassified level. 

5 

None 

6 

The format with hyperlinks is by far the better intelligence product. 

7 

Hyperlinks would not necessarily make the job faster because it could 
take longer for the investigator/employee/administrator to navigate the 
hyperlinks however the hyperlinks would be invaluable to those who 
would choose to use them. 

8 

Information with contact or source information is most relevant to 
ensure quality reporting/actions 

9 

Though the two documents said basically the same thing the second 
one was much easier to follow because 1 could look up what 1 didn't 
know and determine its worth to my organization. 

10 

In my current day-to-day job 1 wouldn't react to this type of info without 
further authority to do so but, if 1 was directed to react, the hyperlinks 
would be very useful since there wasn't enough information in the non- 
hyperlinked info for me to have a good grasp of the situation. 

11 

A hyperlink with the classified ID of the 11 and their MOs. 
a list of potential secondary sites, 
a capability to share with "significant private partners" 
a financial cost/risk column for cross reference and potential 
escalation/de-escalation analysis...(this allows for speculation to future 
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hardening or MOs) 

nice job...this can be translated into other sub disciplines i.e., EMS 
public health and fire intel sharing you may want o speak to Jim Tindall 
502 or I can send you my thoughts about creating a national medical 
intel center which could greatly benefit from this concept 


12 

Hyperlinks can also be used in a similar manner as explanatory 
footnotes in a research paper. Hyperlinks provided further explanation if 
the reviewer was not familiar with the subject, group, tactic, etc. 
referenced in the intelligence report. 

13 

Hyperlinks benefit in a military environment might be of limited value 
unless a cross-domain security solution exists. Much intel analysis is 
done on JWICS and SIPRNET. These systems do not have much 
interconnection with unclass networks. 

14 

1 know it is just a sample but both missed important decision data. What 
region of the state (if available or say it is not). Descriptors for wanted 
subjects (photos if available) and the names of known group members. 

15 

1 feel the hyper links add an added component to the Intel product 
which can be used at the customer needs. 

16 

The use of hyperlinks is a great idea. The initial bulletin is like an 
executive summary. The hyperlinks provide the opportunity for to gain 
more information if needed. 

17 

The hyperlinks provided access to the sources of information that were 
helpful in developing a complete picture with background. 

18 

The hyperlinks made the report more credible and interesting than the 
example without supporting documentation. The only suggestion 1 
would make is to limit the graphics if possible to speed up download 
time. The last hyperlink (FBI 1999 Terrorism Report) took too long to 
download and this may cause some people to be frustrated. 

19 

The problem with hyperlinks is that they need to be kept active. 1 
clicked on one which might not be active on purpose but then 1 would 
have to spend the time to research the link if it was information that 1 
thought was necessary to read or maybe 1 felt well if the links aren't 
active then maybe the report isn't credible and 1 shouldn't bother with it. 

20 

The questionnaire asked for my discipline, and then it removed N/A as 
an answer. Most of the questions regarding intelligence and job 
effectiveness should have been answered with N/A from an emergency 
management perspective, but that answer was not available. 

21 

This looks like a great project! I'll be curious to see the final proposal 
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and recommendations-especially considerations regarding 
classification/distribution, etc. This might not be part of the scope of the 
project, but it would be interesting to have a strategic planning group 
come up with a fact sheet with the information too--the links are great, 
but time constraints may delay reading important information. 

Excellent product. I can't tell you how excited I would be to have a 
report such as to work off of. 

22 

Please push this out as soon as you can. With limited time and 
resources these days, we need all the help we can get. 

I use hyperlinks within documents and pdf's in most of my work. If the 
hyperlink does not exist in the PDF then I place hyperlinks within the 

23 document. Hyperlinks are very important in locating important 
supporting information. However, the hyperlinks MUST be link to the 
proper reference otherwise the link would be misleading. 

The addition of hyperlinks was an improvement. The number of 

24 hyperlinks used, however was distracting. Not all of them worked, and 
took more time to look at than it did to read the article. 

Public Health in this region is VERY rarely included in the intel loop. 

25 This is very different from my former location where the hyperlinked 
version would have been very useful. 

When using a date in the summary, I would include the year to avoid 

26 confusion and allow the reader to quickly see the age of the data before 
deciding to view the ink. 

The use of the hyperlinked information is a real time saver and is the 
obvious choice for the busy manager. My only concern is there may be 
a tendency for the manager to become lazy and rely only on the 
information presented without independent research on his/her part. 

The selection of the most correct and possibly diverse sources of 
hyperlinked information is critical to his/her success. 

27 

From the Emergency Management perspective, the intelligence 
information will allow us to prepare for those events that may not be 
prevented even though we have some intelligence indicating they are a 
possibility in the near future. This is valuable information for the 
emergency manger assuming he/she is cleared to inform those that 
need the information to prepare. 

Great idea - logical, not sure if current SOGs for Intel products already 

28 include this. Would be interesting to know. Some of what I get - FOUO 
stuff - does use hyperlinks. 
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29 What a great simple change. 

30 Thank you... 

The fire service doesn't get or use much in the way of advanced 
intelligence for resource allocation. I have strained my brain trying to 
find an intelligence "niche" for the FS, but it seems to be very small. For 
large departments (such as mine), if the advanced warning time is 

31 sufficient, we could use it for specialized training in the potential target 
facility, but developing this takes time. Adding a list of potential targets-- 
or past target structure types--(e.g., testing laboratories, Federal 
buildings, etc.) for the group involved would be helpful...maybe as a 
hyperlink? 

Great idea to be able to drill down to obtain additional information on 

32 hyperlinked items. How about a color coded system, to be able to 
discern between Names, Organizations, Attacks. 

The second intelligence document with the hyperlinks provided much 
more information, documented credible sources and also allowed for 
the logical flow of the report. It was much better than the first report 
without hyperlinks. The portion of the intelligence product that was 
missing was the SO WHAT, an Assessment from the 
information/intelligence presented in the report and recommendation of 
actions, if any for the user of the product to take as a result of the 
intelligence report. 

Hyperlinks are a great resource tool if you follow a few simple 
guidelines: 1. Hyperlinks must be accurately setup on a key word rather 
than an entire sentence. 2. Hyperlinks must point directly to the 
relevant source. (News Media and many websites move links around 
constantly so what is available during the report may not be available 
after distribution. 3. Keep the number of links down to the most 
important topics, and a brief overview of the source is still needed. 
(Follow an MLA format and there shouldn't be any problems.) 

Hyperlinks better than no links. Kicker is that the hyperlinks are only 
additive if the information at the linked site is useful. It's either GI/GO or 
the 'other guy' has an informative, useful page. Using the examples 
given, predominantly a civilian law enforcement example, most of the 
information available is already provided. Would it truly be beneficial to 

35 know when Mr. San Diego was arrested for car theft when he was 15? 

I must say this is a good effort. My view is from DoD, and my personal 
view is that you kill bad guys, you don't arrest them. This survey is very 
domestic focused, and seems to reinforce the view that 9/11 et al, are 
law enforcement issues. This misses the point that such events are 
perpetrated by foreign, non-GENEVA signatories; they are hostile and 
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lethal, and researching them like a criminal case will, in the long run, 
hurt the Nation. 

I don't know if there is a better answer within the US Homeland. Too 
many lawyers and too much bureaucracy. 

I found the intelligence report with the hyperlinks provided me with 
more information in about the same amount of time as the report 
without the links. I did not see additional information that would have 

36 changed the priorities of my job. I think one of the values I could see 
from the hyperlinks is the ability to build a "living" intelligence report 
with additional information added as it becomes available, resulting in 
more timely and specific information. 

This was a very useful exercise. I do believe that the way information is 
imparted to analysts allows them to collate more effectively; however, it 
is still the quality of the information, rather than just the way it is 
presented, which is most important to the EM aspects of intelligence 
operations. For instance, since EM is a response/planning discipline, 
any operational aspect has to be proactive; so, in order for additional 
value to be added, EOC's have to be activated, teams assembled, 

37 information disseminated...and yet, if the information is valueless or 
without credibility, it makes it all that much more difficult to convince the 
decision makers that a decision needs to be made. The hyperlinks 
were valuable, but I would venture, only from a research viewpoint. 
Second-flight incarnations of this platform could have small "notes" in 
place of the hyperlinks so that one doesn't even have to wait for the 
hyperlinks to activate; as the mouse is moved over the hyperlink, the 
notes immediately pop up. This would certainly make the information 
gathering and situational awareness process more timely. 

38 Loved the use of hyperlinks - more efficient and time-saving for me! 

Since I work in "health" we have more limited access to the background 
information that traditional LE has access to. Therefore, the ability to 
3 Q get additional information via a hyperlink provides good background 
information for review and assistance in considering the potential health 
impacts or need specific to changes in "medical/public health" 
intelligence gathering such as syndromic surveillance etc. 

40 Hyper links embedded in documents is a great idea 

The concern with hypertext links is the tracking of web links by the 
source. While useful, the hypertext links really would just help novices 
close the initial learning gap. For example, SHAC is well known for 
those of us who monitor the AE activities throughout the nation. 

Absent secure links I would be hesitant to visit some hypertext linked 
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sites, especially the unknown websites. 


Special thanks for conducting research in this important area of HSHD. 


42 

1 do not use the type of info reported on in product for my job, but 1 like 
the tool itself, and found it very easy to use 

43 

In my present position the two scenarios would not benefit me. 1 
answered the questions based on me being employed in a position that 
could use this type of intelligence. 

44 

Those of us in Environmental Health who will participate in responding 
in the event of a disaster or Security incident need more information...of 
threats...not just notification of an event. 1 wonder about the security of 
the notification system...how will we prevent it from getting into the 
wrong hands? 

45 

You change your scoring halfway thru the last survey. You start with 6 
meaning N/A, and then it switches to strongly agree. Some may miss 
this transition. It's obvious you're testing the use and viability of 
hyperlinks, not sure you need to ask so many questions to get there. 
Good survey and good luck. 

46 

The resources are easier to use with hyperlinks and are more likely to 
be used. A good start in general. Need more useable information in the 
field. Hyperlinks would be a good idea on all levels, classified or non- 
classified. 

47 

The hyperlinked report allowed for greater authentication, increased my 
view as to credibility and allowed me to "see" the supporting detail and 
concerns for myself. 

48 

one of the links could not be found on the Justice Department server 

49 

The hyperlink allowed me to access information that interested me, and 
quickly get the "meat" of the intel report vs. having to weed through a 
lot of "stuff". The hyperlink provided an opportunity to deeper into the 
topics that interested me, thus allowing me to focus my reading time 
more efficiently. 

50 

Great concept - keep it up. 

51 

This is a great foundation...1 would like to see this with an analytical 
product that identifies trends and threats - and to be completely selfish 
that effect the fire service, hazmat, EMS and Urban Search and 

Rescue. 

52 

The hyperlinked example was familiar to me primarily because of my 


108 




previous use of "wikipedia" - it was easy to use and provided more 
information. 1 thought the second example was definitely a better 
product, easier to use, and provided opportunity for greater knowledge. 

53 

1 like the versatility of the hyperlink, if 1 didn’t know the subject well 1 
could click the hyperlink and get the background 

54 

excellent use of a common tool 

55 

Great idea - potentially very useful. My only concern was info overload - 
while it should improve productivity 1 would have a tendency to delve 
too deeply into the supporting information and might lose sight of the 
focus of the intel product. 

56 

1 found the Hyper Links to be an all around better format. 

57 

Hyper links offer the reader the opportunity to locate more detailed 
information or additional sources of information. Great idea! 

58 

Hyperlinks are useful, but could be more information than needed. We 
operate in an information overload and do not need excess, non useful 
information. Good Intel does not necessarily mean more Intel but it 
needs to be relative to the product I'm using. 

59 

There is an obvious difference in the quality and depth of the two 
products. Unfortunately, 1 have seen way too many of the former and 
not enough of the latter during the period of my career when 1 worked in 
the intelligence arena. The second certainly provides clarification and 
ease of use...obviously; if 1 were still in the field 1 would prefer the 
second product. 

60 

It would be more useful if he links were established as credible reliable 
sources. We get lulled into using hyperlinks not checking the veracity of 
the poster for the hyperlink... assuming the author did that vetting for 
us. Being intelligence professional skepticism runs deep so 1 would 
want to verify not only the initial report but any additional information 
referenced. 1 would caution hyperlinks are not a panacea unless 
standards are established to ensure the entire product (including the 
links) are verified and trusted as being accurate. 

61 

Although intel doesn't always have judgments or conclusions, it is 
certainly more useful that anecdotal information. 1 thought the 
hyperlinks help to develop a better understanding of the information 
that was presented in the sample. 

62 

Until 1 participated in your survey, the Animal Rights activists were just 
marginally part of my perception. You changed my mind with the 
hyperlinks since 1 learned that some of these groups target the nuclear 
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sector. Very useful for a manager who likes to get relevant info quickly, 
with enough depth to be certain that the information is credible, but 
without a lot of keystrokes. 

63 

Hyperlinked product is excellent and easy to use. 

64 

More information is available through the hyperlinks, ease of access, 
ability to quickly gather historical data etc. 

65 

Good-luck with your research. 1 feel strongly that using hyperlinks will 
enhance not only the effectiveness but will also limit "unnecessary" 
time researching documents, i.e.,: statutes, agreements, codes etc. 

66 

The system is as good as the intel 

67 

Hyper-links are much more beneficial to time saving and a quick up to 
snuff. 

68 

One of the hyperlinks did not have the correct information when clicked 
on. 1 believe it was "Kevin Kjonaas." Putting too many hyperlinks could 
cause distraction however, so 1 believe there is a tradeoff between 
providing too much information and timely information. 

69 

If this survey results in hyper-links being included in future intelligence 
briefings, this will have been worth the time spent on the survey and 
would greatly enhance local gov't's ability to plan and respond more 
effectively. 

70 

Hyperlinks will definitely improve knowledge in areas that small 
agencies don't normally have the time to spend on, and add credibility 
to the information being reported. 

71 

None 

72 

It was like night and day! 

73 

The type of hyperlink is more important than just having the links 
available. Some hyperlinks are not valuable or add value to the info 
block. Hyperlinks that help with additional insight, pros/cons, or can 
help link one event with another is most helpful. 

74 

Hyperlinks are the way to go. The take full advantage of today's 
technology. 

75 

Good concept, people will seek out more information when it is easily 
accessible. Perhaps the difference between a more though out, 
researched product versus a mediocre survey of information. 


76 


As a policy maker I don't see much intelligence; however I do see the 
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benefit of having intelligence with additional background information for 
a non-intel/law enforcement policy maker can grasp a better picture of 
the situation. 

77 

Nice concept that allows for greater collaboration 

78 

Would hyperlinked data go through the Internet? If so, how is the 
transmission of classified data to be secured? 

79 

The use of hyperlinked intelligence (AND OTHER REPORTS) is a 
great idea. Hopefully, this will be the way of the future. 

80 

It is clearly better with hyperlinks. It cuts down on investigative 
background time. 

81 

PHOTOS AND PRIMARY UNIT OR POINT OF CONTACT IS ALWAYS 
NEEDED HOWEVER LACKING IN QUITE A FEW INTEL REPORTS. 

AS MUCH AS POSSIBLE ONE LEAD UNIT OR AGENCY SHOULD 

BE IDENTIFIED AS MORE AND MORE WE ARE STEPPING ON 

EACH OTHERS TOES AND OVERLAPPING. 

82 

Great idea! 

83 

Second sample give more knowledge. 

84 

Hyperlinks to large files take some time to download, which breaks the 
continuity of information flow. If the hyperlinks were to edited 
summaries of Intel info, it would facilitate more timely downloads and 
reading. 

85 

1. Being in Emergency Management, intelligence are not a regular 
feature of my job - but they should be. 

2. Hyperlinks are only good if current, and as a report ages, links are 
likely to become inactive. Also, links to a specific item within a web 
page are more efficient than a generic link to a web page - in that you 
then have to spend time searching for the info you want. 

86 

Intelligence analysts are required to sift through voluminous amounts of 
reports and data to develop an assessment. Any tools — such as the 
use of hyperlinks ~ that can be incorporated to facilitate speed of 
analysis should be utilized. 

87 

As an emergency manager my needs for intelligence generally center 
around knowing the where's and when's of a potential incident — the 
sources, methods, leads, and general law enforcement and intel 
background information is of little value to me. 

88 

At first 1 was not sure just what you were asking. 1 may not be on a front 


Ill 



line organization, however, I would find this information important and 
needed. 


89 

The hyperlinked product is superior. It effectively provides the means to 
answer questions thus eliminating subsequent searching. 

90 

none 

91 

The product didn't contain recommendations or conclusions so it was 
difficult to rate the product on those criteria. 

92 

Great idea, and while hyperlinking will require more front-loading of 
information, will significantly increase the usefulness of the product and 
will significantly reduce additional research required by the analyst. 

93 

Your agree....and disagree were at the wrong ends....1 should 
represent Agree. 

94 

none 

95 

None 

96 

Good Luck!!! 

97 

Very illustrative survey - well done! 

98 

Hyperlinks allow for shorter executive summaries as well as more 
details if needed. It also allows for different levels of security. I.e., Exe 
Sum at the FOUO and Hyperlinks at a more secure level. 

99 

Great concept. Look forward to reading the thesis when it is published. 

100 

Honestly, I'd consider all my responses to be outliers and should not be 
considered in your thesis. Reasoning: 1 don't use intelligence products 
in my day to day activities. We create emerging technology solutions to 
support homeland security. The entities 1 support would most likely be 
valuable in guiding and developing algorithms that could be utilized to 
automatically scan and mine large volumes of material to help develop 
the intelligence product that you would communicate to users. 1 would 
have considered prescreening questions at the beginning of the survey 
to include/exclude survey responders based upon their 
need/use/interaction with intelligence products. The survey appears 
that it would be very useful for those it pertains to. Best of luck! 

101 

None 

102 

Be careful of hyperlinking too much...sometimes that is a distracter and 
may drive the reader down the wrong "rabbit hole" as they start 
following links away from the main story/product 


112 



The "hyperlinks" were great! The initial direction (first paragraph) of the 
scenario was so vague, however, that it made it difficult to comment on 
effectiveness. From a Statewide perspective and having 566 

103 municipalities the intelligence dealing with a single meeting within the 
next 2 months anywhere in the State is hard to follow-up on. If the point 
of the survey was to test intelligence with and without the hyperlink - it 
is clear that the hyperlink product is very worthwhile and informative. 

I am a health physicist and basically only care about radiation - the 
hyperlinks would allow me to rapidly determine if there was a history or 
"connection" to radiation. If a suspect had a degree in physics and we 

104 had a report of stolen radioactive material - we might start pulling the 
tread. The hyperlinks would give any body a better chance of seeing a 
connection to their field. Downside - it could take a lot of time to surf 
through all the info. 

Hyperlinks are a valuable tool in being able to reinforce conclusions or 
facts; however, it is important that the links be to credible, known 
sources. I would take some stock into a source that was linked to in a 

,| 0 5 government briefing more than I would by searching Google for 
information on a subject. However, it would be important that the 
briefings only contain credible sources that the governmental agency 
issuing the briefing has validated the information that the sources 
contain. 

lOg Superb use of inteL.the more information provided, the greater 
opportunity for decisions to be made that affect local operations. 

107 Good tool for an alert with the hyperlinks allowing more time for 
research into possible threats in local area 

Hyperlinks are great for providing background information - but it is not 

10 g hyperlinks that are the problem - the problem is the quality of 

intelligence and the availability of actionable information. For example, 
in this example the information is not actionable for my organization. 

109 Hyperlinks provided quick, useful information when needed. 
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Please read this condensed sample unclassified Intelligence Product. Seven questions on the quality and value of this product will be 
asked after you have completed reading this sample. You may come back to this page via the back button at any time. 

SAMPLE - FOR RESEARCH ONLY 

The State Police have received information that environmental terrorists may have scheduled a long term strategic planning session in 
the state within the next 60 days. Please contact your local law enforcement or the FBI Joint Terrorism Task Force with any 
information you may receive concerning this meeting or contacts with known or suspected environmental terrorist group members. 

In March, a closely watched trial in New Jersey ended with the convictions of Stop Huntingdon Animal Cruelty (SHAC)and six of its 
members for using the group's Web site to incite violence against people associated with Huntingdon Life Sciences, a firm that tests 
drugs on animals. Victims, who had personal information about them and their families posted on the Web site, testified that they had 
received death threats, seen their windows smashed and cars overturned, and had their children threatened, among other things. The 
six, who now face up to seven years in prison, were the first ever charged under the Animal Enterprise Protection Act, a law that was 
amended in 2002 to include "animal enterprise terrorism." 

One of those convicted was Kevin Kjonaas, who was president of SHAC until his indictment in the case in 2004. Jurors heard 

U->A o m-.11 «•-, -n^.ol CT™ Tl.o™ U, -ift-o- C™ TlioaA -.Ho-to-IItt frT,m 

* 


zi 


1 startl I ^ © 


Bn ti 1 ‘ 11 Igf] Thesis Draft... | Course: Mult... | .jTj Zoomerang-... 1 1 http://ww... 


< rz)(§> 3:41 fl 


3 http://www.zoomei-ang.com - Zoomerang - Microsoft Internet Explorer 


evidence that Kjonaas had made a call to Daniel Andreas San Diego just hours after San Diego allegedly bombed two California 
businesses in August 2003, although Kjonaas was not charged with any crimes in connection with the bombings. The FBI has offered 
a $250,000 reward for the arrest of San Diego, who has tattoos of burning buildings on his chest. 

On Jan. 19, another major case was brought against 11 people accused of terrorist attacks in Oregon, Wyoming, Washington, 
California and Colorado on behalf of the Animal Liberation Front (ALF) and the Earth Liberation Front (ELF) In a 65-count 
indictment returned by a grand jury in Oregon, the 11, who called themselves "The Family," are accused of 17 attacks, including 
sabotaging a high-tension wire and destroying a Vail, Colo., ski resort, in a $12 million arson. 


3 *The Product was timely and relevant to your mission, programs, 
priorities, or initiatives. 


Strongly 

Disagree 



Strongly Agree 

N/A 

u 

2J 

U 

-4J SJ 

SJ 


4 *The product was clear and logical in the presentation of information with 
supported judgements and conclusions. 



Strongly 


_ 


Disagree 

Strongly Agree N/A 



U 2J 2J 
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5 

*The product is reliable; i.e. sources well documented and reputable. 



Strongly 




Disagree 

Stro n g ly Ag re e N/A 
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Please read this condensed sample unclassified Intelligence Product. Nineteen questions on the quality and value of this product will 
be asked after you have completed reading this sample. You may come back to this page via the back button at any time. 


SAMPLE - FOR RESEARCH ONLY 

The State Police have received information that environmental terrorists may have scheduled a long term strategic planning session in 
the state within the next 60 days. Please contact your local law enforcement or the FBI Joint Terrorism Task Force with any 
information you may receive concerning this meeting or contacts with known or suspected environmental terrorist group members. 

In March, a closely watched trial in New Jersey ended with the convictions of Stop Huntingdon Animal Cruelty (SHAC) (page 80 of 
280) and six of its members for using the group's "Web site to incite violence against people associated with Huntingdon Life 
Sciences , a firm that tests drugs on animals. Victims, who had personal information about them and their families posted on the Web 
site, testified that they had received death threats, seen their windows smashed and cars overturned, and had their children 
threatened, among other things. The six, who now face up to seven years in prison, were the first ever charged under the Animal 
Enterprise Protection Act , a law that was amended in 2002 to include "animal enterprise terrorism." 


One of those convicted was Kevin Kjonaas , who was president of SHAC until his indictment in the case in 2004. Jurors heard 
evidence that Kjonaas had made a call to Daniel Andreas San Diego just hours after San Diego allegedly bombed two California 
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One of those convicted was Kevin Kjonaas , who was president of SHAC until his indictment in the case in 2004. Jurors heard 
evidence that Kjonaas had made a call to Daniel Andreas San Diego just hours after San Diego allegedly bombed two California 
businesses i n August 2003, although Kjonaas was not charged with any crimes in connection with the bombings. The FBI has offered 
a $250.000 reward for the arrest of San Diego, who has tattoos of burning buildings on his chest. 

On Jan. 19, another major case was brought against 11 people accused of terrorist attacks in Oregon, Wyoming, Washington, 
California and Colorado on behalf of the Animal Liberation Front (ALF) and the Earth Liberation Front (ELF) In a 65-count 
indictment returned by a grand jury in Oregon, the 11, who called themselves "The Family," are accused of 17 attacks, including 
sabotaging a high-tension wire and destroying a Vail, Colo., ski resort, in a $12 million arson (page 18 of 68j 




10 The product was timely and relevant to your mission, programs, 
priorities, or initiatives. 


Strongly 

Disagree 



Strongly Agree 
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ii The product was clear and logical in the presentation of information with 
supported judgements and conclusions. 
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Strongly 

Disagree 

Strongly Agree 

N/A 
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The product is reliable; i 

i.e. sources well documented and reputable. 

Strongly 



Disagree 

Strongly Agree 

N/A 
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13 The product would contribute to satisfying intelligence gaps or 
predicating cases or intelligence operations, especially previously 
unknown areas. 


“3 


Strongly 

Disagree 



Strongly Agree 

N/A 

U 

-2J 

SJ 

U 5J 

-SJ 


14 The product would result in a change in investigative or intelligence 
priorities and/or a shift from unaddressed to addressed work, or vice 
versa. 


Strongly 

Disagree 



Strongly Agree 

N/A 

U 

-U 

sj 

-SJ SJ 

-SJ 


13 The product would result in more informed decisions concerning 
investigative or intelligence initiatives and/or resource allocation. 


Strongly 

Disagree 



Strongly Agree 

N/A 

U 

2J 

-SJ 

-U SJ 

-SJ 


16 -The product would identify new information associated with pending 

matters or offered insights into information that could change the working 
premise in a program or initiative 



Strongly 

Disagree 

Strongly Agre 
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21 *Using intelligence products with hyperlinks would make it easier to do 
my job. 


Strongly 

Disagree 

Strongly Agree 

U JJ JJ 4J JJ 

JJ 


*1 would find intelligence products with hyperlinks useful in 

my job. 

Strongly 


Disagree 

Strongly Agree 

U 2J JJ U SJ 

JJ 


23 ’Learning to utilize intelligence products with hyperlinks would be easy for 
me. 


Strongly 

Disagree 




Strongly Agree 

JU 

JJ 

JU 

JU 

JJ JU 


24 *| would find it easy to use intelligence hyperlinks to obtain decision 
making information. 


Strongly 

Disagree 




Strongly Agee 

U 

JJ 

JU 

JU 
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